Network device maintenance

ABSTRACT

A method to access a device may include obtaining, at a first device, data over a short-range wireless network from a second device. The data may originate at a remote system that sends the data to the second device through a network connection over a wide area network. The method may also include in response to a fault at the second device, obtaining, at the first device from the remote system, a maintenance command for the second device. The maintenance command may be obtained by the first device over an analog voice network. The method may also include directing, from the first device to the second device, the maintenance command over the short-range wireless network to enable the second device to perform the maintenance command.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Pat. No. 17,024,519, filed onDec. 19, 2020, now U.S. Pat. No. 10,833,920, which is a continuation ofU.S. patent application Ser. No. 16/712,654, filed on Dec. 12, 2019, thedisclosures of each of which are incorporated herein by reference intheir entireties.

FIELD

The embodiments discussed herein are related to communication oftranscriptions.

BACKGROUND

Audio communications may be performed using different types of devices.In some instances, people that are hard-of-hearing or deaf may needassistance to participate in the audio communications. In theseinstances, transcriptions of the audio may be provided to thehard-of-hearing or deaf. To provide the transcriptions to ahard-of-hearing or deaf person, a particular device or applicationrunning on a mobile device or computer may be used to display texttranscriptions of the audio being received by the hard of hearing ordeaf person.

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one example technology area where some embodiments describedherein may be practiced.

SUMMARY

A method to access a device may include obtaining, at a first device,data over a short-range wireless network from a second device. The datamay originate at a remote system that sends the data to the seconddevice through a network connection over a wide area network. The methodmay also include in response to a fault at the second device, obtaining,at the first device from the remote system, a maintenance command forthe second device. The maintenance command may be obtained by the firstdevice over an analog voice network. The method may also includedirecting, from the first device to the second device, the maintenancecommand over the short-range wireless network to enable the seconddevice to perform the maintenance command.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 illustrates an example environment for transcription ofcommunications;

FIG. 2 illustrates an example environment for transcription ofcommunications;

FIG. 3 illustrates example operations related to accessing a device;

FIG. 4 is a flowchart of an example method to access a device;

FIG. 5 illustrates an example environment for maintenance of a device;

FIG. 6 illustrates an example environment for transcription ofcommunications;

FIG. 7 illustrates an example environment for transcription ofcommunications;

FIG. 8 illustrates an example environment for user monitoring;

FIG. 9A illustrates an example environment for routing audio atranscription;

FIG. 9B illustrates another example environment for routing audio of atranscription;

FIG. 10 illustrates an example environment for communicating atranscription and corresponding audio over a same communication channel;

FIG. 11 illustrates another example environment for communicating atranscription and corresponding audio over a same communication channel;

FIG. 12A illustrates an example environment for training an encodingsystem and a decoding system;

FIG. 12B illustrates an example autoencoder that may be an example ofthe encoding system and the decoding system of FIG. 12A;

FIG. 13 is a flowchart of an example method to communicate atranscription and corresponding audio over a same communication channel;and

FIG. 14 illustrates an example system that may be used duringtranscription of communications.

DESCRIPTION OF EMBODIMENTS

Hard-of-hearing people may use one or more devices with a display duringcommunication sessions to assist their understanding of thecommunication sessions. For example, a transcription of audio of acommunication session may be presented in real-time or substantiallyreal-time on a display of a device of a hard-of-hearing person. As aresult, the hard-of-hearing person may read the words spoken by athird-party during the communication session as well as listen to thewords to achieve better understanding during the communication session.In these and other circumstances, to obtain the transcription of theaudio, the audio of the communication session may be directed to atranscription system during the communication session. The transcriptionsystem may generate the transcription of the audio during thecommunication session and send the transcription to the device forpresentation of the transcription by the device.

Currently, some devices that present transcriptions during communicationsessions using internet protocols (IP) networks connections through aninternet service provider to direct audio to and receive transcriptionsfrom a transcription system for communication sessions conducted overanalog voice network, such as a plain old telephone system (POTS).However, not all heard-of-hearing users have access to an internetservice provider. Some embodiments in this disclosure relate to systemsand methods that may be used to send audio to and receive transcriptionsfrom a transcription system without use of IP network connectionsthrough an internet service provider. For example, in some embodiments,audio may be directed to a transcription system over an analog voicenetwork. For example, the audio may be directed to the transcriptionsystem using bridging such that the audio is directed to thetranscription system by the analog voice network. In these and otherembodiments, the transcription of the audio may be directed back to adevice over a cellular network or by embedding the transcription withthe audio on the analog voice network.

Alternately or additionally, some embodiments of this disclosure relateto systems and methods to set-up and/or manage one or more devices in aresidence of a hard-of-hearing user. For example, a device that obtainstranscriptions over a cellular network may have one or more processes toset-up the device and to maintain the device. Some embodiments in thisdisclosure may disclose how a device may be provided to ahard-of-hearing user with reduced operations in a process to set-up thedevice for operation of the device. Alternately or additionally, someembodiments in this disclosure may discuss how a remote system mayaccess a device with or without an IP network connection through aninternet service provider to help maintain the device.

The systems and methods described in this disclosure may thus providenew and improved systems and methods to provide transcriptions of audioto a device and/or set-up and maintain a device. Furthermore, thesystems and methods described in this disclosure may improve technologywith respect to audio communications and transfer of communicationsbetween devices.

Turning to the figures, FIG. 1 illustrates an example environment 100for transcription of communications. The environment 100 may be arrangedin accordance with at least one embodiment described in the presentdisclosure. The environment 100 may include a network 102, a remotedevice 110, a first device 112, and a transcription system 130.

The network 102 may be configured to communicatively couple the remotedevice 110 and the first device 112. The network may also be configuredto communicatively couple the first device 112 and the transcriptionsystem 130. Alternately or additionally, the network may also beconfigured to communicatively couple the remote device 110 and thetranscription system 130. In some embodiments, the network 102 mayinclude any short-range wireless network, such as a wireless local areanetwork (WLAN), a personal area network (PAN), or a wireless meshnetwork (WMN). For example, the network 102 may include networks thatuse Bluetooth® Class 2 and Class 3 communications with protocols thatare managed by the Bluetooth® Special Interest Group (SIG). Otherexamples of wireless networks may include the IEEE 802.11 networks(commonly referred to as WiFi®), Zigbee networks, Digital EnhancedCordless Telecommunications (DECT) networks, among other types of LANS,PANS, and WMNS. In some embodiments, the network 102 may include anInternet Protocol (IP) based network such as the Internet that isprovided by an Internet service provider (ISP). In some embodiments, thenetwork 102 may include cellular communication networks for sending andreceiving communications and/or data including via hypertext transferprotocol (HTTP), direct data connection, wireless application protocol(WAP), etc. The network 102 may also include a mobile data network thatmay include third-generation (3G), fourth-generation (4G),fifth-generation (5G), long-term evolution (LTE), long-term evolutionadvanced (LTE-A), Voice-over-LTE (“VoLTE”) or any other mobile datanetwork or combination of mobile data networks. In these or otherembodiments, the network may include any combination of analog, digital,and/or optical networks that form a public switched telephone network(PSTN) that may transport audio of a communication session. In these andother embodiments, the portions of the network 102 that communicativelycouple any one of the remote device 110, the first device 112, and thetranscription system 130 to any other of the remote device 110, thefirst device 112, and the transcription system 130 may include one ormore of the network types described above.

Each of the remote device 110 and the first device 112 may be anyelectronic or digital computing device. For example, each of the remotedevice 110 and the first device 112 may include a desktop computer, alaptop computer, a smartphone, a mobile phone, a tablet computer, atelephone, a VoIP (Voice over IP) phone, a phone console, a captiondevice, a captioning telephone, or any other computing device that maybe used for communication between users of the remote device 110 and thefirst device 112.

In some embodiments, each of the remote device 110 and the first device112 may include memory and at least one processor, which are configuredto perform operations as described in this disclosure, among otheroperations. In some embodiments, each of the remote device 110 and thefirst device 112 may include computer-readable instructions that areconfigured to be executed by each of the remote device 110 and the firstdevice 112, respectively, to perform operations described in thisdisclosure.

In some embodiments, each of the remote device 110 and the first device112 may be configured to establish communication sessions with otherdevices. For example, each of the remote device 110 and the first device112 may be configured to establish an outgoing communication session,such as a telephone call, video call, or other communication session,with another device over a telephone line or other network, such as aportion of the network 102. For example, each of remote device 110 andthe first device 112 may communicate over a wireless cellular network, awired Ethernet network, an optical network, and/or a POTS line.

In some embodiments, each of the remote device 110 and the first device112 may be configured to obtain audio during a communication session.The audio may be part of a video communication or an audiocommunication, such as a telephone call. As used in this disclosure, theterm audio may be used generically to refer to sounds that may includespoken words. Furthermore, the term “audio” may be used generically toinclude audio in any format, such as a digital format, an analog format,or a propagating wave format. Furthermore, in the digital format, theaudio may be compressed using different types of compression schemes.Also, as used in this disclosure, the term video may be used genericallyto refer to a compilation of images that may be reproduced in a sequenceto produce video.

As an example of obtaining audio, the remote device 110 may beconfigured to obtain first audio from a first user. For example, theremote device 110 may obtain the first audio from a microphone of theremote device 110 or from another device that is communicatively coupledto the remote device 110. The remote device 110 may be configured todirect, to the first device 112, the audio of a communication sessionbetween the remote device 110 and the first device 112. In these andother embodiments, the first device 112 and/or the remote device 110 mayalso direct the audio to the transcription system 130.

The transcription system 130 may include any configuration of hardware,such as processors, servers, and storage servers that are networkedtogether and configured to perform a task. For example, thetranscription system 130 may include one or multiple computing systems,such as multiple servers that each include memory and at least oneprocessor. The transcription system 130 may be configured to generatetranscriptions from audio.

In some embodiments, the transcription system 130 may be an automaticsystem that automatically recognizes speech independent of humaninteraction to generate the transcription. In these and otherembodiments, the transcription system 130 may include speech enginesthat are trained to recognize speech. The speech engine may be trainedfor general speech and not specifically trained using speech patterns ofthe participants in the communication session. Alternatively oradditionally, the speech engine may be specifically trained using speechpatterns of one or both of the participants of the communicationsession.

Alternatively or additionally, the transcription system 130 may be are-voicing system. In a re-voicing system, a human may listen to theaudio and re-voice or speak the words in the audio. The re-voiced audiomay be provided to a speech recognition system that is trained for thespeech of the human that is re-voicing the audio. In some embodiments,the speech recognition system may listen to the audio of thecommunication session and/or the re-voiced audio. Additionally oralternatively, the speech recognition system may output a transcriptionof the re-voiced audio and/or of the audio without re-voicing. In theseor other embodiments, the transcription system 130 may be a combinationof an interface to a human transcriber and one or more speech engines invarious configurations. For example, a speech engine may create atranscription based on audio of the communication session and a humantranscriber may listen to the same audio and correct the transcription.Additionally or alternatively, the speech engine may create a firsttranscription and the human transcriber may create a secondtranscription and the two transcriptions may be fused into a singletranscription.

In some embodiments, the transcription system 130 may be configured toobtain audio from either the remote device 110 and/or the first device112. In these and other embodiments, the transcription system 130 maygenerate a transcription of the audio. The transcription system 130 mayalso direct the transcription of the audio to the first device 112and/or the remote device 110. Either one or both of the remote device110 and/or the first device 112 may be configured to present thetranscription received from the transcription system 130. For example,the first device 112 may be configured to display the receivedtranscriptions on a display that is part of the first device 112 or adisplay of a device that is communicatively coupled to the first device112. In some embodiments, the transcription system 130 may providecaptions to multiple devices simultaneously. In some embodiments, thetranscription system 130, first device 112, and/or another system maycreate and maintain a record of displays selected to show captions forone or more communication sessions. In instances in which a deviceassociated with a first display is conducting a communication session,it may retrieve the record of displays and send a connect message to oneor more other displays or to a routing system configured to directcaptions to displays.

In some embodiments, the transcription system 130 may be configured toreceive the audio of a communication session between the remote device110 and the first device 112 by having the audio routed through or tothe transcription system 130. For example, in some embodiments, thetranscription system 130 may be configured as an intermediary devicebetween the remote device 110 and the first device 112 such that audioof a communication session between remote device 110 and the firstdevice 112 is routed through the transcription system 130. Variousmethods to have the audio routed through or to the transcription system130 are described with respect to at least FIGS. 9A and 9B.

In some embodiments, the transcription system 130 may be configured toreceive the audio of a communication session between the remote device110 and the first device 112 from either one of the remote device 110and the first device 112. For example, in some embodiments, the firstdevice 112 may send the audio to the transcription system 130 over asecondary network that includes one or more devices. For example, thefirst device 112 may send the audio to the transcription system 130 overan IP based network connection using a router communicatively coupledwith the first device 112.

In some embodiments, the transcription system 130 may be configured toreceive the audio of a communication session between the remote device110 and the first device 112 from a device that obtains the audio of thecommunication session. For example, a device may be positioned betweenthe remote device 110 and the first device 112. The device may obtainaudio of a communication session and direct the audio to thetranscription system 130. An example configuration of a device isdescribed with respect to at least FIGS. 6 and 7 .

As described, the transcription system 130 in response to obtainingaudio of a communication session may generate a transcription of theaudio of the communication session. After generating the transcription,the transcription system 130 may direct the transcription to one or bothof the remote device 110 and the first device 112.

In some embodiments, the transcription system 130 may direct thetranscription to one or both of the remote device 110 and the firstdevice 112 using the same network type over which the transcriptionsystem 130 obtained the audio. For example, the first device 112 maydirect the audio to the transcription system 130 over an IP basednetwork. In these and other embodiments, the transcription system 130may direct the transcription to the first device 112 over the IP basednetwork. As another example, the audio may be directed to thetranscription system 130 over an analog voice network. In these andother embodiments, the transcription system 130 may direct thetranscription to the first device 112 over the analog voice network.Various examples that describe how the transcription may be directed tothe first device 112 over an analog voice network are described withrespect to at least FIGS. 10-13 .

In some embodiments, the transcription system 130 may direct thetranscription to one or both of the remote device 110 and the firstdevice 112 using a different network type than a network type over whichthe transcription system 130 obtained the audio. For example, the audiomay be directed to the transcription system 130 using an analog voicenetwork and the transcription of the audio may be directed to thetranscription system 130 using a separate network. Various examplesregarding the transcription system 130 directing the transcription toone or both of the remote device 110 and the first device 112 using adifferent network type then a network type over which the transcriptionsystem 130 obtained the audio are described with respect to at leastFIGS. 2, 3, and 4 .

As described, one or more of the remote device 110 and the first device112 may communicate with the transcription system 130. To establishcommunications with the transcription system 130, the remote device 110and the first device 112 may include initial configurations that may beused to establish the communications. The initial configurations may bedetermined during an initial use of the remote device 110 and the firstdevice 112. In some embodiments, one or more of the remote device 110and the first device 112 may be provided by an entity that may controlthe transcription system 130. In these and other embodiments, thelocation of the remote device 110 and the first device 112 may bedistributed throughout a region and separate from the transcriptionsystem 130. Thus, in some circumstances, a trained user of the remotedevice 110 and the first device 112 may not have easy access to theremote device 110 and the first device 112 during an initial use of theremote device 110 and the first device 112. In these and otherembodiments, the remote device 110 and the first device 112 may bepre-configured to establish the communications or may be configured toreduce requirements to establish the communications. Various examplesregarding the remote device 110 and the first device 112 beingpre-configured to establish the communications or being configured toreduce requirements to establish the communications are described withrespect to at least FIG. 2 .

Further, in some embodiments, as described above, the location of theremote device 110 and the first device 112 may be distributed throughouta region and separate from the transcription system 130. As a result,when maintenance of the remote device 110 and the first device 112 maybe advised, it may be difficult for a trained user of the remote device110 and the first device 112 to access the remote device 110 and thefirst device 112. In some embodiments, remote maintenance of the remotedevice 110 and the first device 112 may be occur. Various examples ofremote maintenance of the remote device 110 and the first device 112 aredescribed with respect to at least FIGS. 2, 3, and 4 .

Modifications, additions, or omissions may be made to the environment100 and/or the components operating in the environment 100 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the environment 100 may be integrated into otherenvironments that provide additional benefits for a user of theenvironment 100. An example environment that includes the environment100 is provided with respect to at least FIG. 8 .

FIG. 2 illustrates an example environment 200 for transcription ofcommunications. The environment 200 may be arranged in accordance withat least one embodiment described in the present disclosure. Theenvironment 200 may include a first network 202, a second network 204, athird network 206, a remote device 210, a first device 212, a seconddevice 214 and a transcription system 230.

In some embodiments, the first network 202, the remote device 210, firstdevice 212, and the transcription system 230 may be analogous to thenetwork 102, the remote device 110, the first device 112, and thetranscription system 130, respectively, of FIG. 1 . Accordingly, nofurther explanation is provided with respect thereto. Generally, thesecond device 214 in conjunction with the second network 204 and thethird network 206 may be configured to communicatively couple the firstdevice 212 and the transcription system 230.

In some embodiments, the second device 214 may be configured to relaydata between the first device 212 and the transcription system 230 usingthe second network 204 and the third network 206. In these and otherembodiments, the second device 214 may be any electronic or digitalcomputing device. For example, the second device 214 may include arouting device, a network connection device such as a hotspot device orhub, a desktop computer, a laptop computer, a smartphone, a mobilephone, a tablet computer, or any other computing device that may be usedto relay data. In these and other embodiments, the second device 214 mayinclude memory and at least one processor, which may be configured toperform operations as described in this disclosure, among otheroperations. In some embodiments, the second device 214 may includecomputer-readable instructions that are configured to be executed by thesecond device 214 to perform operations described in this disclosure.

In some embodiments, the first device 212 and the second device 214 mayeach include an electrical connection. Alternately or additionally, thefirst device 212 and the second device 214 may share an electricalconnection. In these and other embodiments, a power converter with asingle connection to alternating current power may include two directcurrent (DC) power outlets. One of the DC power outlets may be providedto the first device 212 and another to the second device 214. In theseand other embodiments, a data connection between the first device 212and the second device 214 may be established using the powerconnections. Alternately or additionally, the cable that conducts thepower connections may also include a data cable that may be used toestablish a data connection between the first device 212 and the seconddevice 214. In these and other embodiments, the data connection may beused in place or concurrently with the second network 204.

In some embodiments, the first device 212 may supply power to the seconddevice 214. In these and other embodiments, the power may be derivedfrom the first device 212 via a splitter attached to the powerconnector, a USB port, a headset port, line power (from the cableentering the phone), or another connection to first device 212. In someembodiments, the first device 212 or the cable entering the phone maysupply power to the second device conditioned on a set of criteria,which may include a stipulation that the first device 212 and/or thesecond device 214 is configured to receive transcriptions from thetranscription system 230. In these or other embodiments, in response tothe set of criteria not being met, a power port or connector may bedeactivated so that it does not provide power. Additionally oralternatively, the first device 212 may be configured to indicatewhether a power port is active and ready to supply power to the seconddevice 214. For example, the first device 212 may be configured toilluminate a panel light to indicate that a power port is active andready to supply power to the second device 214. Alternately oradditionally, in place of supplying power to operate the second device214, the first device 212 may supply power to charge a battery of thesecond device 214 that supplies the power to operate the second device214. In these and other embodiments, the power supplied by the firstdevice 212 may provide additional power during operation or when morepower is needed. In some embodiments, the second device 214 may supplypower to the first device 212 in a manner similar to how the firstdevice 212 may supply power to the second device 214 as described above.

In some circumstances, depending on the design of the second device 214,the second device 214 may operate when a battery module is inserted intothe second device 214 and functioning. In these and other embodiments,the second device 214 may be at risk of failing if the battery modulefails, even though the second device 214 may still receive externalpower. As a remedy, a battery simulator may be inserted into the seconddevice 214 in place of a battery module. The battery simulator maybehave as a real battery module and may appear to the second device 214as if the second device 214 included a functioning battery module. As aresult, the second device 214 may continue to function without concernof the battery module failing.

In some embodiments, the battery simulator may be powered by a chargingcurrent provided by the second device 214 and may return voltages orsignals back to the simulator that appear to indicate that a functionalbattery module is operating in the second device 214. In a firstexample, a resistor voltage divider may derive power from two pinsdesigned to receive power for charging a battery cell and may provide avoltage, selected to indicate a working battery module, via one or moresensor pins, back to the hotspot. In a second example, the batterysimulator may transmit a set of signals via one or more sensor pins thatindicate to the second device 214 that a battery module is active andfunctioning properly. In a third example, a battery simulator may beconstructed using electronics similar to that of a real battery module,but where the battery cell or cells are replaced by a circuit thatsimulates the battery cell(s). In a fourth example, the batterysimulator may send a voltage or other signal via one or more connectorsto imitate action of a thermistor that may be used the second device 214to determine actions of a battery module.

In some embodiments, the second network 204 may include a short-rangecommunication network. In some embodiments, the second network 204 mayinclude a short-range wireless communication network, such as a wirelesslocal area network (WLAN), a personal area network (PAN), or a wirelessmesh network (WMN). For example, the network 102 may include networksthat use Bluetooth® Class 2 and Class 3 communications with protocolsthat are managed by the Bluetooth® Special Interest Group (SIG). Otherexamples of wireless networks may include the IEEE 802.11 networks(commonly referred to as WiFi®), Zigbee networks, Digital EnhancedCordless Telecommunications (DECT) networks, among other types of LANS,PANS, and WMNS.

In some embodiments, the second network 204 may be configured tocommunicatively couple the first device 212 and the second device 214.In these and other embodiments, the second network 204 may be configuredto transfer audio of a communication session that occurs between theremote device 210 and the first device 212. The second network 204 maytransfer the audio between the first device 212 and the second device214. Alternately or additionally, the second network 204 may beconfigured to transfer transcriptions of audio of a communicationsession that occurs between the remote device 210 and the first device212 that are generated by the transcription system 230. The secondnetwork 204 may transfer the transcriptions between the first device 212and the second device 214. The second network 204 may also be configuredto transfer other data between the first device 212 and the seconddevice 214.

In some embodiments, the second network 204 may be controlled by thefirst device 212. Alternately or additionally, the second network 204may be controlled by the second device 214 or some other device in theenvironment 200. In these and other embodiments, the second device 214may grant the first device 212 access to the second network 204 based oncredentials supplied by the first device 212 to the second device 214.The first device 212 may obtain the credentials using one or moremethods.

In some embodiments, the first device 212 may obtain credentials toaccess the second network 204 based on information stored in the firstdevice 212. For example, the first device 212 may be manufactured toinclude the credentials to access the second network 204. In these andother embodiments, the first device 212 may include particularcredentials that are set based on credentials for the second device 214.

As an example, upon boot-up, the first device 212 may determine if thefirst device 212 has previously been configured. In response to noprevious configuration or in response to not gaining access to thesecond network 204, the first device 212 may scan available accesspoints. The first device 212 may obtain information from the scans ofthe multiple access points. For example, the information may be aservice set identifier (SSID) or other information. When the SSID orother information of a found access point matches an entry in a table ofthe first device 212, the first device 212 may use stored credentialsassociated with the matching stored entry to initiate a connection toaccess the second network 204. In these and other embodiments, the firstdevice 212 may determine multiple access points that provide informationthat matches an entry in a table. In these and other embodiments, thefirst device 212 may provide the stored credentials to the access pointsuntil access is granted to the second network 204. Alternately oradditionally, the first device 212 may request input from a user todetermine the access point to use to obtain access to the second network204.

In some embodiments, the first device 212 may obtain credentials toaccess the second network 204 based on requesting information. In theseand other embodiments, the first device 212 may request information froma user of the first device 212, another device coupled to the secondnetwork 204, and/or another system, such as the transcription system230.

For example, the first device 212 may be configured to requestinformation from a user. In these and other embodiments, the firstdevice 212 may obtain information from the user that may be used toaccess the second network 204. The information may include one or moreof an identifier of the second network 204 such as the SSID and apassword. In some embodiments, the first device 212 may obtain theidentifier and present the identifier. In these and other embodiments,the first device 212 may request the user to select the identifier andinput the password.

In some embodiments, the first device 212 may obtain information fromanother device connected to the second network 204 that may be used bythe first device 212 to access the network. Alternately or additionally,the first device 212 may obtain the information from another system. Inthese and other embodiments, the first device 212 may have previouslyprovided the information to the system. Alternately or additionally, thesystem may include part or all of the information. In these and otherembodiments, the first device 212 may provide identifying information,such as the SSID of the network or other information about the firstdevice 212 to the other system. The other system may determine theremaining information for the first device 212 to access the secondnetwork 204 and provide the remaining information to the first device212. In these and other embodiments, the first device 212 maycommunicate with the other system using the first network 202, forexample using dual-tone multi-frequency (DTMF) signaling over an analogvoice network.

As another example, both the first device 212 and the second device 214may obtain an identifier from another system, such as the transcriptionsystem 230. In these and other embodiments, the first device 212 mayobtain the identifier through the first network 202 and the seconddevice 214 may obtain the identifier through the third network 206. Inthese and other embodiments, the first device 212 may be configured toprovide the identifier with a connect message to the second device 214.The second device 214 may be configured to provide access to the secondnetwork 204 for those devices that provide a connect message with theidentifier. As such, the first device 212 may access the second network204.

The third network 206 may include a wide area network. In someembodiments, the third network 206 may include an Internet Protocol (IP)based network such as the Internet that is provided by an Internetservice provider (ISP). In some embodiments, the third network 206 mayinclude cellular communication networks for sending and receivingcommunications and/or data including via hypertext transfer protocol(HTTP), direct data connection, wireless application protocol (WAP),etc. Alternately or additionally, the third network 206 may also includea mobile data network that may include third-generation (3G),fourth-generation (4G), fifth-generation (5G), long-term evolution(LTE), long-term evolution advanced (LTE-A), Voice-over-LTE (“VoLTE”) orany other mobile data network or combination of mobile data networks.

In some embodiments, the third network 206 may be configured tocommunicatively couple the second device 214 and the transcriptionsystem 230. In these and other embodiments, the third network 206 may beconfigured to transfer audio, transcriptions, and other data between thesecond device 214 and the transcription system 230. In some embodiments,the third network 206 may be controlled by a wireless telecommunicationsprovider or some other network provider.

An example of the operation of the environment 200 is now provided. Insome embodiments, a communication session between the remote device 210and the first device 212 may be established such that audio originatingat the remote device 210 is directed to the first device 212 over thefirst network 202. The first device 212 may present the audio for a userof the first device 212. The first device 212 may also direct the audioto the second device 214 over the second network 204. The second device214 may direct the audio to the transcription system 230 over the thirdnetwork 206. The transcription system 230 may generate a transcriptionof the audio and direct the transcription to the second device 214 overthe third network 206. The second device 214 may direct thetranscription to the first device 212 over the second network 204.

In some embodiments, an amount of data shared between the transcriptionsystem 230 and the second device 214 may be reduced. In someembodiments, the data shared between the transcription system 230 andthe second device 214, such as the audio and/or transcriptions, may bereduced by compressing the data. Alternately or additionally, the datamay be reduced by reducing an amount of the data. For example, not allof the audio obtained by the first device 212 may be directed to thetranscription system 230. Rather, silence or other portions of audio forwhich transcriptions are not to be generated may not be directed overthe third network 206.

In some embodiments, an amount of data shared between the transcriptionsystem 230 and the second device 214 may be reduced by not sending audioand transcriptions of the audio over the third network 206. For example,in some embodiments, audio of a communication session may be obtained bythe transcription system 230 over the first network 202. Variousexamples of how the audio may be obtained by the transcription system230 over the first network 202 are described with respect to at leastFIGS. 9A and 9B. In these and other embodiments, transcriptions of theaudio generated by the transcription system 230 may be directed to thefirst device 212 over the third network 206.

In some embodiments, an amount of data shared between the transcriptionsystem 230 and the second device 214 may be reduced by not sending allof the audio. In these and other embodiments, the first device 212obtains audio of a communication session. The first device 212 may sendthe audio through a filter to extract ASR (automatic speech recognition)features of the audio. The ASR features may include the aspects of theaudio that may be used by an ASR system to generate a transcription ofthe audio. For example, the ASR features may include LSFs (line spectralfrequencies), cepstral features, and MFCCs (Mel Scale CepstralCoefficients), among other features. Additional information, such asamplitudes of a speech waveform measured at a selected samplingfrequency may also be included with the ASR features. The first device212 may direct the ASR features to the transcription system 230. Thetranscription system 230 may use the ASR features to generate the audiofor re-voicing by a human or directly for generating a transcription ofthe audio.

In some embodiments, an amount of data shared between the transcriptionsystem 230 and the second device 214 may be reduced by using the thirdnetwork 206 in response to the unavailability of another method for thefirst device 212 to direct audio to the transcription system 230 andobtain transcriptions from the transcription system 230. For example,the first device 212 may direct audio and obtain transcriptions overanother network. In response to the other network not functioningproperly, the first device 212 may direct audio and obtaintranscriptions over the third network 206. In some embodiments, theamount of data shared between the transcription system 230 and thesecond device 214 may be based on an estimate of the available bandwidthin an available network. For example, in response to network bandwidthbeing determined (e.g., estimated) as satisfying a bandwidth thresholdthat may be based on a relatively high amount of bandwidth, the data maybe uncompressed or compressed by a relatively small amount. For example,if the bandwidth from the transcription system 230 to the second device214 is sufficient to transmit the audio and transcription in theiroriginal forms, the data may be uncompressed. For example, if the audiois encoded in a 64 kb/s format, and the transcriptions are generated ata peak rate of 200 bits/second, then the threshold may be set at 64.1kb/s. In this example, if the network bandwidth is 200 kb/s (greaterthan the threshold) then the audio and data may be transmitteduncompressed. In contrast, in response to the network bandwidth beingdetermined (e.g., estimated) as not satisfying the bandwidth threshold,the data may be compressed. In these or other embodiments, the amount ofcompression may increase as the estimated amount of bandwidth decreases.Data may be compressed, for example, by using a speech compressionmethod such as code excited linear prediction (CELP), MP3, Opus, FLAC(Free Lossless Audio Codec), Speex, mu-Law, G.711, G.729, GSM, etc.

In some embodiments, the second device 214 may be configured to accessother systems besides the transcription system 230. For example, thethird network 206 may be a general mobile data network that isconfigured to access the Internet. As a result, any device able toaccess the second network 204 may be able to direct data to the seconddevice 214 for transmission over the third network 206. In these andother embodiments, one or more methods to secure the second device 214,the second network 204, and/or the first device 212 may be employed tohelp to prevent unauthorized use of the second device 214 to direct dataover the third network 206. Various example of methods to secure thesecond device 214, the second network 204, and/or the first device 212are now provided.

In some embodiments, the second device 214 may include a unique passwordto access the second network 204 and thereby be configured to directtraffic over the third network 206.

In some embodiments, the second device 214 may limit an amount of datathat may be transmitted over the third network 206. In these and otherembodiments, data limits may be capped at a level corresponding to theuse that may occur for sending audio to and/or obtaining transcriptionsfrom the transcription system 230. Alternately or additionally, a datarate for sending and receiving data over the third network 206 may becompared to a threshold at the maximum rate needed for sending audioand/or obtaining transcriptions. In response to the data rate exceedingthe threshold, the data rate may be capped at a threshold. Alternatelyor additionally, in response to the data rate exceeding the threshold analert may be generated and sent to a service system or the data rate maybe permitted up to a selected volume of data within a selected period oftime. Alternately or additionally, inspection of the data may occur todetermine if the data corresponds to data provided to and/or obtainedfrom the transcription system 230. In response to the data notcorresponding to the data provided to and/or obtained from thetranscription system 230, the second device 214 may stop transmittingthe data, generate an alert, or perform other methods described in thisdisclosure.

In these and other embodiments, the first device 212 may be configuredto access the second device 214 to determine if another device is usingthe second device 214 to access the third network 206. Use of the seconddevice 214 may include attempts to impair operation or to use the seconddevice 214 to provide Internet service to an unauthorized device. Thesecond device 214 may determine if another device is using the seconddevice 214 to access the third network 206 based on comparing settingsof the second device 214 to known settings, including hashed passwords,and checking usage patterns such as data transfer rates and total usageover a particular period of time. In these and other embodiments, thefirst device 212 may perform the comparison and checking or anothersystem, such as the transcription system 230 may perform the comparisonand checking. For example, the second device 214 may determine that theamount of data sent to or from the third network 206 in a selectedperiod of time exceeds a selected threshold and then send an alert to amonitoring system or otherwise act to report or block the inappropriateusage. In response to an indication that another device is using thesecond device 214 to access the third network 206, the first device 212may log the evidence found and the configuration, disable the seconddevice 214, reconfigure the second device 214, change credentials forthe second device 214, request instructions for actions to perform fromanother system, among others.

In some embodiments, disabling the second device 214 may includedisabling the second device 214 in a manner such that the first device212 may reenable the second device 214. For example, the first device212 may provide a code to the second device 214 to reenable the seconddevice 214 in response to input from a user of the first device 212and/or an indication that the inappropriate usage of the second device214 has stopped.

Alternately or additionally, disabling the second device 214 may includedisabling the second device 214 in a manner such that the first device212 and/or a user associated with the first device 212 may not enablethe second device 214. In these and other embodiments, the second device214 may be enabled by sending the second device 214 to the manufactureor provider of the second device 214. Alternately or additionally, thesecond device 214 may be enabled using a particular tool that ismaintained by authorized agents of a service associated with thetranscription system 230. Alternately or additionally, the second device214 may be enabled through use of a particular password, configurationupdate, or other firmware update that may be obtained in response to arequest but that is not available to the user of the first device 212.

Another method to secure the second device 214, the second network 204,and/or the first device 212 may include locking the second device 214 sothat a password used to access the second network 204 maintained by thesecond device 214 cannot be read or changed by a user of the seconddevice 214 or the first device 212. Locking the second device 214 mayalso prevent reading information from the second device 214 and makingconfiguration changes to the second device 214. In these and otherembodiments, the password may be changeable, for example, only via aservice set identifier (SSID) set by the second device 214 for thesecond network 204 and a password that may be unique to the seconddevice 214 or set via one or more remote commands from a service system.Alternately or additionally, the SSID and password may be known by thefirst device 212 so that the first device 212 may login to and makeconfiguration changes to the second device 214.

Alternately or additionally, the second device 214 may be fully lockedso that login and configuration changes are impossible except byconnecting the second device 214 to specialized equipment that changesthe firmware of the second device 214. Alternately or additionally, thesecond device 214 may be fully locked unless accessed with certainpasswords or unpublished actions such as pressing two unrelated buttonsat once.

Alternately or additionally, specific configuration parameters of thesecond device 214 may be partly locked so that “safe” functions (e.g.check data rate/signal strength, check connectivity status, read logs,reset to a default or operational state, and functions related tooperability of the second device 214) are available. However, in thisand other embodiments other functions (e.g. read/modify SSID, password,or MAC address tables, adding a new client, factory reset) may be lockedso that the functions cannot be read and/or modified. Examples ofconfiguration parameters may include a master password, username andpassword, a list of MAC addresses or other device identification codesto specify devices authorized to connect to the second device 214 usingthe second network 204, Internet service usernames and passwords, SIMcodes, IP addresses, wireless settings such as SSID and passwords orsecurity keys, logs, and DHCP settings, among others. In someembodiments, the second device 214 may have two or more sets of logincredentials, including, for example, a username and/or password, eachwith a different level of access. For example, a first level of accessmay enable a first set of configuration parameters to be read but notmodified and a second set of configuration parameters to be modified. Asecond level of access may enable a third set of configurationparameters to be read but not modified and a fourth set of configurationparameters to be modified. A given set of credentials may be known tothe first device 212, a service system or remote service center such asa help desk, the subscriber or user, an authorized, and/or or equipmentused by the installer.

The first device 212 may be programmed with credentials, such as an SSIDand password, to login to the second network 204 and access the seconddevice 214. The credentials may be unavailable to a user of the firstdevice 212 and known only to the first device 212. The credentials maybe stored in the first device 212 in an encrypted form that may bedecrypted with a key provided by an authorized installer or in a messagefrom a service system of the first device 212. In these and otherembodiments, in response to a change of the credentials of the seconddevice 214, the first device 212 may be unable to access the secondnetwork 204. In response to being unable to access the second network204, the first device 212 may alert a service system that may deactivatethe second device 214.

In some embodiments, the second device 214 may send a signal to aservice system of the second device 214, directly or via the firstdevice 212, if the second device 214 is reset, if the password is reador changed, or if a new device is connected to the second device 214. Inresponse to such a signal, the service system may act to discontinueservice or deactivate the second device 214.

In some embodiments, the second device 214 may be remotely monitored andmaintained by a service system for suspect activity such as reset,password access, unauthorized logins (e.g. by WiFi devices other thanthe captioned phone), excessive minutes of use, behavior and usagepatterns that suggest fraud or other misuse, among others. In responseto detection of suspicious activity, further monitoring or investigationmay be implemented or the second device 214 may be deactivated. In theseand other embodiments, the service system may log into the second device214 (e.g. via a browser-accessible monitor/control interface),reconfigure the second device 214, change the password, lock or unlockthe second device 214, reset the second device 214, turn the seconddevice 214 on or off, etc. In some embodiments, the second device 214may be configured to hide the SSID of the second network 204.Alternately or additionally, the service system may monitor the seconddevice 214 via the third network 206 or via the first device 212, todetect and/or correct failures of the first device 212, failures of thesecond device 214, network failures, connection failures, and othercommunication issues such as high packet loss, transmission errors, orreduced or fluctuating bandwidth.

In some embodiments, the second device 214 may be configured so that areset may not be performed or may be performed only with a password. Inthese and other embodiments, the password to reset the second device 214may be different than the password for the second network 204.Alternately or additionally, the second device 214 may cease to functionor connect to the third network 206 in response to a reset or thepassword for the second network 204 may not change in response to areset of the second device 214. In some embodiments, the second device214 may be configured so that, if it is power cycled or factory reset(e.g., by holding a reset button for 10 seconds), it may default to astate where access is restricted, for example where one or more of theconfiguration parameters cannot be read and/or modified.

In some embodiments, the second device 214 may include a whitelist ofdevice identifiers, such as a media access control (MAC) address, thatthe second device 214 may allow to access the second network 204. Thus,if a device does not include a device identifier on the whitelist thesecond device 214 may deny access to the device. In these and otherembodiments, the first device 212 may include a device identifier thatis on the whitelist to allow the first device 212 to access the secondnetwork 204. Alternately or additionally, the whitelist may only includea portion of the device identifiers. In these and other embodiments, inresponse to a device including the matching portion of the deviceidentifier then the device may be allowed to join the second network204. For example, multiple devices may be configured with MAC addressesthat contain a first string (e.g. XX:XX:XX:XX, for example,“12:3D:C8:90”) that is shared among the devices and a second string(e.g. AA:BB) that is unique to the device, so that the full MAC addressappears as, for example, 12:3D:C8:90:AA:BB. The second device 214 mayprovide service to devices where the MAC address includes the firststring (12:3D:C8:90). In these and other embodiments, the second device214 may include the whitelist. Alternately or additionally, anothersystem may include the whitelist. In these and other embodiments, thesecond device 214 may communicate with the other system before grantinga device access to the second network 204.

In some embodiments, the second device 214 may be configured to onlygrant a single device access to the second network 204 and/or grant asingle device access to direct traffic over the third network 206. Insome embodiments, the second device 214 may be configured so thepassword or other settings may be changed according to a particularsetting. Alternately or additionally, the second device 214 may only beconfigured by a particular device or devices, such as the first device212. Alternately or additionally, the second device 214 may beconfigured to allow only certain data or certain types of data such asaudio and transcriptions to be transmitted over the third network 206.Alternately or additionally, the second device 214 may be configured toonly direct network traffic to a particular destination address oraddress such that a request to direct traffic to a system other than thetranscription system 230 may be denied.

In some embodiments, access to the second device 214 may be through apassword obtained from another system using an authentication process.For example, another device may communicate with a service system usinga secure connection. The device may provide login credentials for thedevice and information about the second device 214 to the servicesystem. The information may include information regarding an accountassociated with the transcription system 230, the first device 212 (e.g.MAC address, IP address, serial number, etc.), and the second device214, such as the configuration or identity of the second device 214including a serial number, an IP address, or other identifier such as anidentifier used by the second device 214 to access the third network 206such as a subscriber identity module (SIM) number or an internationalmobile equipment identify (IMEI) number. Using the information, theservice system may direct the password to the device. The password maybe a global password or unique to the second device 214. The device mayuse the password to login to the second device 214. In these and otherembodiments, the login of the device to the second device 214 may be viaa secure link so that a person or machine monitoring communicationbetween the device and the second device 214 may intercept onlyencrypted messages. The device may make changes to the second device214. In these and other embodiments, the changes may be determined bythe device or based on information provided by the service system. Inthese and other embodiments, the device may not store the password ormay delete an internal copy of the password after it is used. In theseand other embodiments, the device used to obtain the password from theservice system may be the first device 212.

In some embodiments, the second device 214 may use a SIM card or IMEI toaccess the third network 206. In these and other embodiments, the SIMcard may be configured to not be removed from the second device 214,such as by securing the SIM card in place so that the SIM card or thesecond device 214 may be damaged by removal or by electronicallyrendering the SIM card inoperable if it is removed. For example, the SIMcard may be secured with an adhesive or an adhesive may be used to holdthe second device 214 or a compartment within the second device 214closed.

Alternately or additionally, the SIM card may include an identifier ofthe second device 214. In these and other embodiments, the SIM card maynot function without obtaining the identifier. Alternately oradditionally, a network service managing an account associated with theSIM card may be configured with an identifier of the SIM card, such asan international Mobile Subscriber Identity (IMSI) and the identifier ofthe second device 214, such as an International Mobile EquipmentIdentity (IMEI). The SIM card identifier and the identifier of thesecond device 214 may be provided to the network service. The networkservice may compare the received identifiers to those in an accountassociated with the SIM card. In response to a match of the identifiers,the network service may provide the second device 214 access to thethird network 206. Otherwise, the network service may not allow thesecond device 214 to access the third network 206.

Alternately or additionally, in response to no match between theidentifier of the SIM card and the identifier of the second device 214,an alert may be generated. The alert may be generated by an alertdetection system. The alert detection system may be part of thetranscription system 230 or some other system. In some embodiments, inresponse to an alert, the SIM card may be automatically disabled suchthat the SIM card may not be used to communicate over the third network206. In these and other embodiments, the SIM card may be disabled bycommunicating with the associated wireless telecommunications providerand indicating that the SIM card is to be disabled. Alternately oradditionally, in response to an alert the second device 214 may beautomatically disabled such that the second device 214 may not relaydata. In these and other embodiments, a communication may be provided tothe second device 214 to disable the second device 214. Alternately oradditionally, services provided by the transcription system 230 may bedisabled such that the first device 212 may not obtain transcriptionsfrom the transcription system 230 of audio of communication sessions.For example, an account associated with a user associated with the firstdevice 212 and the second device 214 may be suspended such thattranscriptions are not generated for the user.

In some embodiments, in response to an alert, a message regarding thealert may be generated and provided to a user associated with the firstdevice 212 and the second device 214. In these and other embodiments,the message may indicate that improper use of the SIM card is occurringor has occurred. The message may be provided using any communicationmedium including, phone calls, emails, letters, text messages,presentation on the first device 212, among others.

In some embodiments, in response to an alert, the SIM card and/or thesecond device 214 may be scheduled to be automatically disabled in thefuture after a particular period of time elapses. In these and otherembodiments, a message regarding the alert and the particular timeperiod may be generated and provided to a user associated with the firstdevice 212 and the second device 214. In response to the particularperiod of time elapsing, the SIM card and/or the second device 214 maybe disabled.

In some embodiments, in response to the SIM card no longer requestingaccess to the third network 206 through unauthorized devices and/or theSIM card requesting access to the third network 206 through the seconddevice 214, the alert may be voided. In response to the alert beingvoided, the services previously disabled may be reenabled. Thediscussion of an alert being generated, and actions taken in response tothe generation of the alert may be applied to other embodimentsdescribed in the disclosure. For example, an alert may be generated inresponse to any indication of improper use of the second device 214and/or SIM card as described in this disclosure.

As described, the second device 214 may be used to provide the firstdevice 212 access to the third network 206. In some circumstances,either one of the first device 212 and/or the second device 214 may notoperate properly. For example, the first device 212 may lose theconnection to the second network 204 and not be able to restore theconnection. Alternately or additionally, the second device 214 maymalfunction such that the first device 212 does not have access to thethird network 206. For example, the connection between the second device214 and the third network 206 may fail and/or the second device 214 maynot properly maintain the second network 204 to allow the first device212 to communicate with the second device 214. Alternately oradditionally, the one or more of the first device 212 and the seconddevice 214 to may need to be reset, reconfigured with new or additionalcodes, settings, firmware, encryption keys, passwords, IP addresses,other data, and/or have other maintenance functions performed. Othersettings that may be maintained in the first device 212 and/or thesecond device 214 may include firewall settings, parameters forcommunicating with network host devices, and firmware updates, etc.

In some embodiments, the first device 212 may be configured to providedirection to the second device 214 to maintain the second device 214. Inthese and other embodiments, and the first device 212 may be configuredto provide one or more maintenance commands to the second device 214.The maintenance commands may originate from the first device 212. Forexample, the first device 212 may include instructions with respect tomaintaining the first device 212. In these and other embodiments, thefirst device 212 may execute the instructions and in response toexecuting the instructions, the first device 212 may send themaintenance commands to the second device 214.

Alternately or additionally, the maintenance commands may originate froma remote system, such as a service center. In these and otherembodiments, the remote system may be a part of, associated with, orindependent from the transcription system 230. In these and otherembodiments, the first device 212 may obtain the maintenance commandsfrom the remote system over the first network 202 and/or the secondnetwork 204. For example, the maintenance commands may be provided overthe first network 202 and/or the second network 204 using standardover-the-air (OTA) wireless delivery. When the second device 214 is notfunctioning such that the first device 212 is not able to receive thedata over the second network 204, the first device 212 may obtain themaintenance commands over the first network 202. In these and otherembodiments, the maintenance commands may be provided to the firstdevice 212 over the first network 202 using DTMF signaling over ananalog voice network.

To provide the maintenance commands using the DTMF signaling over theanalog voice network, the first device 212 may be contacted in a mannersimilar to a communication request for a communication session from theremote device 210. In these and other embodiments, the communicationrequest may result in the first device 212 providing an indication ofthe contact, such as by ringing. In these and other embodiments, thefirst device 212 may behave differently in response to receiving acommunication request for maintenance commands than when receiving acommunication request for a communication session, such as a phone call.

For example, when the first device 212 receives a communication requestfrom the remote system, the first device 212 may determine anorigination address and/or contact information of the communicationrequest, such as a phone number or Caller ID, is associated with theremote system. In response to determining the origination address and/orcontact information is associated with the remote system, the firstdevice 212 may not provide an indication of the communication request ormay wait to provide the indication until the first device 212 determinesthat the origination address and/or contact information is associatedwith the remote system.

In these and other embodiments, the first device 212 may not provide theindication of the communication request in response to the determiningthe origination address and/or contact information is associated withthe remote system and based on one or more other parameters. Forexample, based on a preference of a user of the first device 212, a timeof day when the communication request is obtained, a day of the weekwhen the communication request is obtained, or other criteria may changewhether the first device 212 may not provide the indication of thecommunication request.

In some embodiments, the maintenance commands may enable the remotesystem to remotely access the second device 214. Through the remoteaccess of the second device 214, the remote system may perform themaintenance of the second device 214. In some embodiments, the remotesystem may provide maintenance commands for maintaining the seconddevice 214 in response to failure by the first device 212 to maintainthe second device 214. For example, the first device 212 may attempt tomaintain the second device 214 based on instructions stored in the firstdevice 212. When the first device 212 fails to maintain the seconddevice 214, the remote system may provide maintenance commands.

In some embodiments, the maintenance commands may include runningdiagnostics, obtaining a device status, obtaining configurationinformation, setting configuration settings, performing a reset,maintenance or changing settings of a firewall, reconfiguring/updatingpasswords, reconfiguring network addresses, maintaining or settingparameters of the second network 204 and/or the third network 206,maintaining or setting parameters for connection by the second device214 to a remote system such as the transcription system 230, updatingfirmware, among others commands that may be performed to help maintainor restore functionality of a device.

In some embodiments, the first device 212 may provide instructions to auser of the first device 212 and the second device 214 regardingmaintenance of the second device 214. For example, the first device 212may present instructions either by audio or display regardingmaintenance functions to perform with respect to the second device 214.For example, the first device 212 may instruct the user to reset thesecond device 214, power-off and power-on the second device 214, amongother maintenance commands. As another example, the first device 212 mayobtain configuration settings and/or credentials for the second device214 from a user. The first device 212 may provide the configurationsettings and/or credentials to the user or to the second device 214.Alternately or additionally, the first device 212 may provide a servicenumber that may be used to establish a communication session with a helpservice for maintaining the second device 214.

In some embodiments, when the second device 214 is not operatingcorrectly, the first device 212 may alert a user of the first device 212regarding the status of the second device 214. In these and otherembodiments, the second device 214 may send alerts to the user through adisplay, by playing audio, or sending a message to another device of theuser. For example, a display on the first device 212 may alert the userthat a connection to the third network 206 has been lost, is not stable,or lacks sufficient bandwidth to provide captions.

The maintenance commands may be issued in response to routinemaintenance or release of updates for the second device 214. Alternatelyor additionally, the maintenance commands may be issued in response to afault in the second device 214. A fault in the second device 214 mayinclude connectivity of the second device 214 to the one or more of thesecond network 204 and the third network 206. Alternately oradditionally, a fault in the second device 214 may include theconnectivity of the second device 214 to a remote system, such as thetranscription system 230. Alternately or additionally, a fault in thesecond device 214 may include the connectivity of the second device 214to the first device 212.

Alternately or additionally, a fault in the second device 214 mayinclude the connectivity of the first device 212 to the transcriptionsystem 230 through the second device 214. For example, the second device214 may include an issue such that the second device 214 may notcorrectly pass data between the first device 212 and the transcriptionsystem 230 along the second network 204 and the third network 206.Alternately or additionally, a fault in the second device 214 mayinclude other inoperability or maintenance issues of the second device214. For example, a fault in the second device 214 may include thesecond device 214 not including the latest version of firmware,applications, drivers, operating system, or other software.

In some embodiments, a fault in the second device 214 may be determinedby the second device 214, the remote system, and/or the first device212. In these and other embodiments, the second device 214 may determinea fault in the second device 214 based on connectivity issues of thesecond device 214, self-diagnostic of the second device 214, or throughan indication from another device or system. In these and otherembodiments, the first device 212 may determine a fault in the seconddevice 214 based on connectivity of the first device 212. For example,in response to the first device 212 being unable to identify, connect,or otherwise interact with the second device 214 through the secondnetwork 204, the first device 212 may determine a fault in the seconddevice 214. Alternately or additionally, in response to the first device212 being unable to ping the transcription system 230 or other systemsthrough the third network 206, the first device 212 may determine afault in the second device 214.

Alternately or additionally, in response to the results of diagnosticsrun on the second device 214 and obtained by the first device 212, thefirst device 212 may determine a fault in the second device 214.Alternately or additionally, in response to the second device 214 notoperating correctly as identified by the first device 212 not obtainingdata from the second device 214 when data is expected. For example, insome embodiments, in response to the first device 212 directing audio tothe transcription system 230, the second device 214 may expecttranscriptions from the second device 214. In response to not obtainingtranscriptions from the second device 214, the first device 212 maydetermine a fault in the second device 214. A remote system, such as thetranscription system 230 may determine a fault in the second device 214in a manner analogous to how the first device 212 may determine a faultin the second device 214.

Modifications, additions, or omissions may be made to the environment200 and/or the components operating in the environment 200 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the second device 214 may be incorporated into the firstdevice 212. In these and other embodiments, the second network 204 maybe an electrical connection between the first device 212 and the seconddevice 214 incorporated into the first device 212. As another example,the functionality of the second device 214 may be included in the firstdevice 212.

As another example, the environment 200 may include another network. Theother network may communicatively couple the second device 214 and thetranscription system 230. In these and other embodiments, the seconddevice 214 may use either the third network 206 or the other network todirect data to and obtain data from the transcription system 230.Alternately or additionally, the other network may communicativelycouple the first device 212 and the transcription system 230. In theseand other embodiments, the first device 212 may direct data to andobtain data from the transcription system 230 over the other network orthrough the second network 204, the second device 214, and the thirdnetwork 206.

In some embodiments, the first device 212 may select to use the othernetwork or second network 204, the second device 214, and the thirdnetwork 206, referred to in this embodiment as the combined networkbased on one or more factors. For example, the factors may include thenetwork with the better or worse connection speed, reliability, or cost,among other factors. In these and other embodiments, the first device212 may use the combined network and the other network. For example, thefirst device 212 may use the combined network for directing data to thetranscription system 230 and may use the other network for obtainingdata from the transcription system 230. As another example, the firstdevice 212 may use the other network for directing data to and obtainingdata from the transcription system 230 and may use the combined networkfor maintenance of the second device 214 or the second network 204 orthe third network 206 or vice versa.

In short, the environment 200 may include multiple other networks thatmay be selected among one or more of the devices or systems, such as thefirst device 212, the second device 214, and the transcription system230 as described in this disclosure.

For example, the environment may include one or more additional networksthrough which the transcription system 230 may communicate with thesecond device 214. In these and other embodiments, the third network andthe additional networks may each be wireless data network, such as a 3G,4G, LTE, or 5G data network that are maintained by different wirelesstelecommunications providers. In these and other embodiments, the seconddevice 214 may include a multi-network SIM card or multiple SIM cards toaccess the third network 206 and the additional networks.

In some embodiments, a determination may be made regarding which of thethird network and additional networks to use for communication betweenthe transcription system 230 and the second device 214. In these andother embodiments, the first device 212, the second device 214, or acombination of the first device 212 and the second device 214 may makethe determination.

In some embodiments, the determination of which network to use may bebased on one or more criteria. For example, the criteria may include asignal strength, upload connection speeds, download connection speeds,cost of data transmission, performance statistics for communicationbetween the transcription system 230 and the second device 214, amongother criteria. In these and other embodiments, performance statisticsfor communication between the transcription system 230 and the seconddevice 214 may include a percentage of time communication is available;a number, frequency, and/or length of interruptions of communication;and performance with respect the particular data being transmitted, suchas the transmission of audio to the transcription system 230 andtransmission of transcriptions to the second device 214.

In some embodiments, the determination of which network to use may bebased on evaluating the networks individually. In these and otherembodiments, if one of the networks does not meet a particularthreshold, a different network may be selected. Alternately oradditionally, the determination of which network to use may be based ona comparison among the different networks. For example, the comparisonamong the different networks may be made based on scoring for each ofthe network. In these and other embodiments, each of the differentcriteria for each of the networks may be assigned a score. The score foreach network may be a sum of the scores for each of the criteria. Inthese and other embodiments, the network with the highest score may beselected for use.

In some embodiments, the determination of which network to use may beperformed at different intervals or continuously. For example, theintervals may be a particular or random time period; a particular orrandom number of communication sessions or portions of communicationsessions, such as portions of a communication session separated bysilence such that data is provide transmitted over one or more of thethird network 206 and other networks; a particular or random amount ofdata exchanged between the transcription system 230 and the seconddevice 214; or some other interval. As a result, in some embodiments,the selected network may change between communication sessions and/orduring a single communication session. In these and other embodiments,when the selected network changes during a communication session, thechange may occur during a period when data exchanged between thetranscription system 230 and the second device 214 is not occurring,such as during periods of silence or when the remote device is notproviding audio with speech.

As another example, in some embodiments, the second device 214 may useone or more of the third network 206 and the additional networks inoverlapping time periods to perform the data exchange with thetranscription system 230. For example, in response to one of the thirdnetwork 206 and the additional networks not providing sufficientbandwidth, multiple of the third network 206 and the additional networksmay be employed.

FIG. 3 illustrates example operations 300 related to accessing a device.The operations 300 may be arranged in accordance with at least oneembodiment described in the present disclosure. In the illustratedexample, the operations 300 may be between a remote device 310, a firstdevice 312, a second device 314, and a transcription system 330. In someembodiments, the remote device 310, the first device 312, the seconddevice 314, and the transcription system 330 may be analogous to theremote device 210, the first device 212, the second device 214, and thetranscription system 230, respectively, of FIG. 2 . Accordingly, nofurther explanation is provided with respect thereto. Alternatively oradditionally, the operations 300 may be an example of the operation ofthe elements of the environment 100 of FIG. 1 .

In some embodiments, the operations 300 may be an example ofcommunications and interactions between the remote device 310, the firstdevice 312, the second device 314, and the transcription system 330. Insome embodiments, the interactions between the remote device 310, thefirst device 312, the second device 314, and the transcription system330 may occur over one or more networks. For example, some of theinteractions may occur over a first network, others may occur over asecond network, and others may occur over a third network. In these andother embodiments, the first network may be an analog voice network, thesecond network may be a short-range wireless network, and the thirdnetwork may be a wide area network.

Generally, the operations 300 may relate to accessing a device. In theseand other embodiments, accessing the device may include accessing adevice to direct maintenance commands thereto where the device iscommunicatively coupled with another device and together configured togenerate transcriptions of audio communications. The operations 300illustrated are not exhaustive but are merely representative ofoperations 300 that may occur. Furthermore, one operation as illustratedmay represent one or more communications, operations, and/or dataexchanges.

At operation 340, the remote device 310 may send audio over the firstnetwork. The audio may be obtained by the first device 312 and by thetranscription system 330. In some embodiments, the audio may be firstobtained by the first device 312. In these and other embodiments, thefirst device 312 may direct the audio to the transcription system 330.Alternately or additionally, the audio may be first obtained by thetranscription system 330. In these and other embodiments, thetranscription system 330 may direct the audio to the first device 312.Alternately or additionally, a system that is part of or coupled to thefirst network may obtain the audio from the remote device 310 and directthe audio to the first device 312 and the transcription system 330. Theaudio may be part of a communication session between the remote device310 and the first device 312. In these and other embodiments, the audiomay include words spoken by user of the remote device 310.

At operation 342, the first device 312 may present the audio to a userof the first device 312. In these and other embodiments, presenting theaudio may include broadcasting the audio via a speaker.

At operation 344, the transcription system 330 may generate atranscription of the audio. For example, the transcription may includethe words included in the audio that are spoken by the user of theremote device 310.

At operation 346, the transcription system 330 may direct thetranscription to the second device 314 over the second network. In theseand other embodiments, the transcription system 330 may include datathat associates the second device 314 with the first device 312. Thus,instead of directing the transcription directly to the first device 312,the transcription system 330 may direct the transcription to the seconddevice 314.

At operation 348, the second device 314 may direct the transcriptionfrom the transcription system 330 to the first device 312 over the thirdnetwork. In these and other embodiments, the second device 314 may routethe transcription to the first device 312 without changing thetranscription.

At operation 350, the first device 312 may present the transcription. Inthese and other embodiments, the first device 312 may present thetranscription such that the presentation of the transcription issubstantially aligned with the presentation of the audio.

At operation 352, the first device 312 may determine a fault in thesecond device 314. The first device 312 may determine the fault in thesecond device 314 based on data received from the second device 314.Alternately or additionally, the first device 312 may determine thefault in the second device 314 based on not receiving data from thesecond device 314. In these and other embodiments, the first device 312may infer the fault in the second device 314.

At operation 354, the first device 312 may direct a notification of thefault of the second device 314 to the transcription system 330 over thefirst network. In these and other embodiments, the notification mayinclude an indication of the fault of the second device 314.

At operation 356, in response to obtaining the notification, thetranscription system 330 may direct one or more maintenance commands forthe second device 314 to the first device 312 over the first network. Inthese and other embodiments, the maintain commands may be directed overthe first network using a DTMF signaling or other analog signaling thatmay be used on an analog voice network.

At operation 358, the first device 312 may direct the maintenancecommands from the first device 312 to the second device 314 over thesecond network. In these and other embodiments, the transcription system330 may not direct the maintenance commands to the second device 314over the third network. In these and other embodiments, thetranscription system 330 may not direct the maintenance commands to thesecond device 314 over the third network because the fault may affectdata exchanged between the second device 314 and the transcriptionsystem 330 over the third network.

At operation 360, the second device 314 may direct a response to themaintenance commands to the first device 312 over the second network. Atoperation 362, the first device 312 may direct the response to themaintenance commands to the transcription system 330 over the firstnetwork.

Modifications, additions, or omissions may be made to the operations 300without departing from the scope of the present disclosure. For example,the operations 300 may not include the operations 360 and 362 in someembodiments. In these and other embodiments, one or more operationsassociated with the operation 352 may be omitted or performed by adevice different than the devices and/or systems indicated in FIG. 3 .For example, the transcription system 330 may determine a fault in thesecond device 314. In these and other embodiments, the transcriptionsystem 330 may direct the maintenance commands to the first device 312over the first network without operations 352 and 354.

As another example, in some embodiments, the operations 300 may bearranged in a different order or performed at the same time. Forexample, operations 344 and 342 may be performed at the same time.Further, the operations 342, 344, 346, 348, and 350 may be performed onan ongoing basis during the communication session. In these and otherembodiments, the operations 342 and 344, 346, and 348 may be performedin substantially overlapping time periods.

As another example, additional operations may exist. For example, thefirst device 312 may direct the audio to the transcription system 330during the operation 340. Alternately or additionally, if a fault isdetermined during a communication session between the remote device 310and the first device 312, the first device 312 may not obtain themaintenance commands until after the termination of the communicationsession.

In some embodiments, the transcription system 330 may obtain themaintenance commands from the first device 312. In these and otherembodiments, the first device 312 may direct the maintenance commands tothe transcription system 330. The transcription system 330 may providethe maintenance commands to the second device 314 over the thirdnetwork. In these and other embodiments, the second device 314 mayprovide a response to the maintenance commands to the transcriptionsystem 330 that may be relayed to the first device 312 over the firstnetwork. The maintenance commands may be sent to the second device 314by way of the transcription system 330 in response to a fault in thesecond network such that communication between the first device 312 andthe second device 314 over the second network is not available.

FIG. 4 is a flowchart of another example method 400 to access a device.The method 400 may be arranged in accordance with at least oneembodiment described in the present disclosure. The method 400 may beperformed, in some embodiments, by a device or system, such as the firstdevice 112 of FIG. 1 , the first device 212 of FIG. 2 , the first device312 of FIG. 3 , or the computing system 1400 of FIG. 14 , or anotherdevice. In these and other embodiments, the method 400 may be performedbased on the execution of instructions stored on one or morenon-transitory computer-readable media. Although illustrated as discreteblocks, various blocks may be divided into additional blocks, combinedinto fewer blocks, or eliminated, depending on the desiredimplementation.

The method 400 may begin at block 402, where data is obtained at a firstdevice over a short-range wireless network from a second device. Thedata may originate at a remote system that sends the data to the seconddevice through a network connection over a wide area network. In someembodiments, the data may be a transcription of audio obtained by thefirst device over the analog voice network during a communicationsession between the first device and a remote device. In someembodiments, the short-range wireless network may be a personal areanetwork or an 802.11 network. In these and other embodiments, the widearea network may include one or more of: a cellular network, a digitalnetwork, and an optical network. In these and other embodiments, theanalog voice network may be a plain old telephone system network.

At block 404, in response to a fault at the second device, one or moremaintenance commands for the second device may be obtained at the firstdevice from the remote system. The maintenance commands may be obtainedby the first device over an analog voice network. In some embodiments,the fault at the second device may include an issue with respect to thenetwork connection over the wide area network between the remote systemand the second device. In these and other embodiments, the issue withrespect to the network connection over the wide area network between theremote system and the second device may be a failure of the networkconnection. In some embodiments, the fault may be detected by the remotesystem.

At block 406, the maintenance commands may be directed from the firstdevice to the second device over the short-range wireless network toenable the second device to perform the maintenance commands. In someembodiments, the maintenance commands may relate to one or more of thefollowing: parameters for connection over the wide area network,firewall settings, firmware updates, resetting commands, configurationsettings, and settings of the short-range wireless network, amongothers.

It is understood that, for this and other processes, operations, andmethods disclosed herein, the functions and/or operations performed maybe implemented in differing order. Furthermore, the outlined functionsand operations are only provided as examples, and some of the functionsand operations may be optional, combined into fewer functions andoperations, or expanded into additional functions and operations withoutdetracting from the essence of the disclosed embodiments.

For example, in some embodiments, the method 400 may further includedirecting, from the first device to the remote system, the audio by wayof the short-range wireless network, the second device, and the widearea network. In these and other embodiments, the remote system may beconfigured to generate the transcription using the audio.

In some embodiments, the method 400 may further include in response toproviding the maintenance commands to the second device, obtaining aresponse from the second device with respect to the maintenancecommands. In these and other embodiments, the method 400 may furtherinclude directing the response to the remote system over the analogvoice network.

In some embodiments, the method 400 may further include detecting, bythe first device, the fault in the second device and providing anindication of the fault to the remote system over the analog voicenetwork. In these and other embodiments, the maintenance commands may beobtained in response to providing the indication of the fault to theremote system.

FIG. 5 illustrates an example environment 500 for maintenance of adevice. The environment 500 may be arranged in accordance with at leastone embodiment described in the present disclosure. The environment 500may include a network 502, a remote device 510, a device 512, and asupport system 520.

In some embodiments, the network 502, the remote device 510, and thedevice 512, may be analogous to the network 102, the remote device 110,and the first device 112, respectively, of FIG. 1 . Accordingly, nofurther explanation is provided with respect thereto. Generally, thesupport system 520 may be configured to provide maintenance support tothe device 512.

The support system 520 may include any configuration of hardware, suchas processors, servers, and storage servers that are networked togetherand configured to perform a task. For example, the support system 520may include one or multiple computing systems, such as multiple serversthat each include memory and at least one processor. The support system520 may be configured to provide maintenance support to the device 512.

In some embodiments, maintenance support may be provided to the device512 in response to the occurrence of one or more events. Various eventsare described below. In some embodiments, the support system 520 maydetect that the device 512 is disconnected or otherwise not operational.The detection may be based on error messages, loss of connectivity, aparticular period of time that the device 512 is not used, or the device512 failing to return a signal. The signal may be a signal that isexpected to be received at particular intervals or a signal in responseto a request signal provided to the device 512. In some embodiments, thedetection that the device 512 is disconnected or otherwise not operatingmay be performed by a transcription system. In these and otherembodiments, the device 512 may detect that the device 512 is not ableto communicate with the support system 520 or other systems such as atranscription system.

In some embodiments, a user of the device 512 may report a problemeither via the device 512 or another device. For example, the user maypress a “help” or similar button or icon on the device 512 that ispreprogrammed to launch diagnostic and correction processes. Alternatelyor additionally, a communication request may be made using the device512 to technical support. In response to the communication request,maintenance support may be provided to the device 512. In these andother embodiments, the maintenance support may be provided during thecommunication request, after the communication request, or in place of acommunication session. Alternately or additionally, a field technicianthat may be interacting with the device may indicate that maintenance isrequired.

In response to a maintenance request, maintenance of the device 512 mayoccur. In some embodiments, the device 512 may automatically try toconnect and fix the problem. In some embodiments, the device 512 maywait for a time period when the user is unlikely to be using the device512. For example, the device 512 may open a maintenance session with thesupport system 520 at night. The device 512 may use any form ofcommunication, such as IP packet-based communication, analog basedcommunication, such as DTMF, or other digital based communication tocommunicate with the support system 520. In these and other embodiments,if the user interacts with the device 512 or the device 512 receives acommunication request, the device 512 may terminate the maintenancesession and try again later.

In some embodiments, the device 512 may automatically advise the supportsystem 520 (via voice to a human agent or digitally to an IVR or otherautomated system) of what the device 512 knows about why maintenanceand/or installation is needed. Alternately or additionally, the device512 may automatically advise the support system 520 regarding thecurrent status of the device 512, for example, no network access,transcription system heartbeat lost, firmware or model update failed anddata regarding the potential maintenance such as reorder tones, callerhung ups, busy tones, error messages, audio or events captured whentrying to call the transcription system, the ISP service, log and/ortranscript and/or audio files from the period of interest, network logininformation, etc. The support system 520 may diagnose the problem eithervia systems used by a human or automatically.

In some embodiments, the support system 520 may diagnose or correctproblems with the device 512 through remote control of the device 512.The remote control of the device 512 may include screen sharing, andassuming full control of the device 512. In these and other embodiments,the support system 520 may see the display of the device 512 as thenavigated by the user, the support system 520, or another user. In theseand other embodiments, the support system 520 may reboot the device 512using a soft or hard reboot, authenticate the device 512 or the user,remotely set up the device 512 (e.g. enter SSID, password, and otherconfiguration parameters) to connect to a network such as WiFi, runnetwork diagnostics, check WiFi signal strength, measure networkbandwidth and stability, run device diagnostics, examine logs, check orconfigure software to handle any firewalls, check or update the softwareversion of the device 512, view the screen, and view/edit configurationoptions. The device 512 may acknowledge commands received from thesupport system 520 and may return any completion results messages orerror messages.

In some embodiments, the remote control may enable the support system520 to draw or point on the screen of the device 512. In these and otherembodiments, the support system 520 may swipe, click, drag, etc., andperform other actions the user of the device 512 could perform. Forexample, the support system 520 may virtually press buttons on thedevice 512 and may virtually go off-hook or on-hook, as if the user hadlifted or replaced the handset. In these and other embodiments, where anIP based connection is not established between the device 512 and thesupport system 520, the remote control may be established with thedevice 512 via an alternate data connection such as described withrespect to FIG. 2 or 3 .

In some embodiments, a communication session between the device 512 andthe support system 520 may allow a support agent (human or IVR) to talkto the user face-to-face via video. The agent may also be able todisplay instructions on the screen for the user or (using thesubscriber's camera) get a visual on things like the information stickeron a modem. The video call may be via the device 512 (over a networkconnection or data over a voice connection) or via a separate device. Insome embodiments, the user may send a picture with information to thesupport system 520. In these and other embodiments, the support system520 may use optical character recognition (OCR), bar code scanning, orother automated image analysis to read the information and configure thedevice 512 accordingly. Alternately or additionally, image analysis andsetup may be accomplished by the device 512.

In some embodiments, the device 512 may be set up or authenticated witha browser that may access a web site. The browser may be triggered bythe user entering a URL, by software in the device 512 responding to adetected problem or a request form the user, by a signal from thesupport system 520, etc. Once the browser is connected, the user may beable to request, set up, or cancel services, manager his/her account,view or change configuration parameters or preferences, diagnoseproblems, make purchases of products and services such as thoseadvertised on the screen. In some embodiments, the device 512, browser,or remote-control application may provide an option for the user to ratethe maintenance.

In some embodiments, the maintenance may include functions such aslogging into the device 512, reading logs, detecting additional loginsto the device 512, reading minutes of use, reading MB of data used,resetting the device 512, changing the password, etc.

In some embodiments, logs stored by the device 512 or a transcriptionsystem associated with the device 512 may include a record of how manyaudio packets have been lost and other statistics regarding the networkconnection quality and stability, either in the communication pathbetween the remote device 510 and the device 512 or between the device512 and the transcription system or both. Logs may include errormessages from the device 512, transcription center, ASR provider,network, network router, etc.

In some embodiments, permission may be obtained to perform themaintenance. In these and other embodiments, when the support system 520initiates a request, a screen display or audio prompt may be providedfor a user to grant permission. Alternately or additionally, the device512 may grant permission or the device 512 or a website may collectaccess permission from a user in advance and, when the support system520 asks for permission, the device 512 may retrieve the previouspermission decision from the user. In some embodiments, the maintenancemay be fully automated, such as with an IVR communicating with DTMF orother data connection, under control of a support agent, or acombination thereof.

If remote maintenance is unsuccessful, the device 512 and/or the supportsystem 520 may open a trouble ticket, connect a human tech supportagent, and/or automatically schedule a field technician visit. If theproblem fits a pattern suggesting there is a software or hardware bug,the support system 520 may automatically submit a ticket to qualityassurance or to development for a software fix or update or for furtherinvestigation. For example, if automatic and/or remote installation ormaintenance processes ultimately fail due to a failed configurationevent, failed heartbeat, or failed diagnostic test, the support system520 may connect a human agent to the user or may automatically schedule,request, or suggest (to the user) an installer/tech support visit. Inthese and other embodiments, the support system 520 may also schedule aninstaller or connect a human agent during setup if the user does notknow the SSID or password or if the device 512 fails to configureautomatically. Alternately or additionally, the support system 520 mayalso schedule a call from the support system 520 or a visit from aninstaller if the device 512 fails to automatically register a particularperiod of time after it is shipped to a new user. Alternately oradditionally, if the device 512 is idle for a period of time (e.g. 6months), the support system 520 may attempt to contact the user (e.g.,by phone, email, text, etc.), diagnose the problem, and/or schedulerecovery of the device 512.

In some embodiments, the maintenance of the device 512 may not beperformed using an IP based network. In these and other embodiments, thedevice 512 may place a telephone call to the support system 520. Thedevice 512 may communicate with the support system 520 using tones orother signals sent in the audio channel such as via DTMF, for example.The device 512 may send the support system 520 data and receive data inresponse. In these and other embodiments, the telephone calls may beplaced at night or when the user is otherwise not using the device 512.In these and other embodiments, if the user interacts with the device512 that indicates that the user is attempting to establish acommunication session, the device 512 may immediately drop the telephonecall and attends to the user's new communication session.

Alternately or additionally, the support system 520 may place atelephone call to the device 512 in response to the support system 520determining to provide maintenance to the device 512. The device 512 maybe programmed to auto-answer and not ring when receiving a telephonecall from the support system 520.

In some embodiments, the device 512 and/or the support system 520 mayperiodically connect to discover if anything is amiss with the device512. For example, the device 512 may call the support system 520 andtransmit an account number, user identifier, and/or previous networkidentification number (such as a telephone number, username, or otherdevice identifier) so that the support system 520 may inspect thecurrent network identification (as detected, for example, using ANI) anddetermine if the network identification has changed and update a recordcontaining the network identification.

Modifications, additions, or omissions may be made to the environment500 and/or the components operating in the environment 500 withoutdeparting from the scope of the present disclosure.

For example, in some embodiments, the support system 520 may beconfigured to diagnose and correct problems associated with a devicecommunicatively coupled with the device 512. For example, the device maybe analogous to the second device 214 of FIG. 2 . In these and otherembodiments, the support system 520 may diagnose problems based oninformation obtained from the device. The information may be obtaineddirectly from the device or by way of the device 512. In these and otherembodiments, the information may include signal strength of wirelessconnections of the device, such as 802.11 connection or a cellularconnection; a log of wireless connection failures; a log of wirelessconnections; a list of devices currently or previously wirelesslyconnected with the device; data usage history; usage time; among otherinformation. Based on the information, the support system 520 maydetermine maintenance that may be performed on the device to assist withthe problem of the device. In these and other embodiments, the supportsystem 520 may communicate directly with the device to perform themaintenance and/or the device 512 may communicate with the device.Alternately or additionally, the maintenance of the device described maybe performed by the device 512 without involvement by the support system520.

FIG. 6 illustrates an example environment 600 for transcription ofcommunications. The environment 600 may be arranged in accordance withat least one embodiment described in the present disclosure. Theenvironment 600 may include a first network 602, a second network 604, aremote device 610, a first device 612, a transcription system 630, and atap system 640. The tap system 640 may include a first network system642, a second network system 644, and a display 646.

In some embodiments, the first network 602, the remote device 610, andthe transcription system 630 may be analogous to the network 102, theremote device 110, and the transcription system 130, respectively, ofFIG. 1 . Accordingly, no further explanation is provided with respectthereto. The first network 602 may be configured to communicativelycouple the remote device 610 and the tap system 640. In these and otherembodiments, the first network 602 and the tap system 640 may beconfigured to communicatively couple the remote device 610 and the firstdevice 612 such that the remote device 610 and the first device 612 mayestablish communication sessions therebetween.

The second network 604 may include a wide area network that maycommunicatively couple the transcription system 630 to the tap system640. The second network 604 may also include one or more short-rangecommunication networks. In these and other embodiments, the short-rangecommunication networks may include a wireless local area network (WLAN),a personal area network (PAN), or a wireless mesh network (WMN).

The first device 612 may be any device that may be used forcommunication between users of the remote device 610 and the firstdevice 612. In some embodiments, the first device 612 may be configuredto operate using an analog voice network, such as a POTS network. Inthese and other embodiments, the first network system 642 maycommunicate with the first network 602 and the first device 612 over ananalog voice network. In these and other embodiments, the first networksystem 642 may be configured to obtain audio directed to the firstdevice 612 from the remote device 610 and direct the audio to the secondnetwork system 644 and the first device 612. The first network system642 may also be configured to direct audio obtained from the firstdevice 612 to the remote device 610. The first network system 642 mayinclude an echo canceller and/or other systems to perform the operationsdescribed in this disclosure. For example, the first network system 642may extract and inject DTMF for one or both of the remote device 610 andthe first device 612.

In some embodiments, the echo canceller may subtract a signaloriginating from the first device 612 from the audio traveling from thefirst network 602 to the first network system 642 to obtain an estimateof the signal originating from the remote device 610 and may providethis estimate to the second network system 644. In these or otherembodiments, the echo canceller may perform the function of a telephonehybrid or a two-wire to four-wire converter and may include one or moretransformers, amplifiers, active and passive analog components, A/D andD/A converters, and software such as digital signal processing software.

In some embodiments, the tap system 640 may further include a microphonethat may collect ambient audio from the user. The ambient audio may beused by the echo canceler (either alone or in combination with audiofrom the first device 612) to obtain audio directed to the first device612. For example, ambient audio and audio from the first device 612 mayeach be filtered, added together, and subtracted from audio from thefirst network system 642 and direct the difference to the second networksystem 644. Alternately or additionally, the tap system 640 may useblind source separation, a method using, for example, principlecomponents analysis, independent components analysis, or neuralnetworks, to separate audio from the remote device 610 from audio fromthe first device 612 and then send separated audio from the remotedevice 610 to the transcription system 630. The blind source separationmay use, as input, previous audio from the current session and/orprevious sessions from one or more callers to train the blind sourceseparation system.

The second network system 644 may be configured to obtain the audio fromthe first network system 642 and direct the audio to the transcriptionsystem 630 over the second network 604. The second network system 644may also be configured to obtain transcriptions of the audio from thetranscription system 630 and direct the transcriptions to the display646. The display 646 may be configured to present the transcriptions ofthe audio. In these and other embodiments, the display 646 may presentthe transcriptions substantially synchronized and/or in substantiallyreal-time with the presentation of the audio by the first device 612.Alternately or additionally, the display 646 may be configured topresent information about a communication session such as busy, ringing,answered, male voice, female voice, laughter, music, etc.

In some embodiments, the display 646 may be a user interface that may beconfigured to obtain user input. Alternately or additionally, the tapsystem 640 may be controlled using DTMF tones from the first device 612.For example, a tone corresponding to a number 1 may enable transcriptionof the audio, a tone corresponding to a number 2 may disabletranscription of the audio, etc.

In some embodiments, the display 646 may be separate from the tap system640. In these and other embodiments, the tap system 640 may communicatewith the display 646 over a short-range communication network. Thedisplay 646 may be part of another electronic device that maycommunicate with the tap system 640. For example, the display 646 may beincluded in an iPad, smart TV, custom display, touch screen, non-touchscreen, a smartphone, software on the user's smartphone or computer thatprints captions on the computer monitor, a smart speaker with a screensuch as Alexa Show, a display on a landline or other phone, a display ina car, a videophone, etc. In these and other embodiments, the tap system640 may function as a line interceptor that connects into a wallconnector on one side and the line plug of the first device 612 on theother and connects the two. In some embodiments, the tap system 640 mayfunction as an ATA (analog telephone adapter), converting betweendigital signals to/from the first network 602 and analog signals to/fromthe first device 612. In these or other embodiments, the ATA may alsoextract audio signals from a digital signal arriving from the remotedevice 610 and send them to the transcription system 630.

In some embodiments, the tap system 640 may gain power from theconnections with the first network 602 and/or the second network 604,phone microphone bias voltage (a.k.a. phantom power), or otherconnection to the tap system 640, either directly as it is needed or tocharge a battery.

In some embodiments, the tap system 640 and the first device 612 mayperform functions similar to that of the first device 112 and/or thefirst device 212 of FIGS. 1 and 2 . For example, the tap system 640 andthe first device 612 may provide access to user settings andpreferences, may call a support system when problems occur with thefirst device 612, the first network 602, the second network 604, and/orthe tap system 640. Alternately or additionally, the tap system 640and/or the first device 612 may operate to perform maintenance andconfigurations as described with respect to this disclosure.

Modifications, additions, or omissions may be made to the environment600 and/or the components operating in the environment 600 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the tap system 640 may be coupled to the first device 612between the body and a receiver or handset of the first device 612.

In some embodiments, the tap system 640 may be constructed withpermanently attached line cords on one or both ends or with no attachedline cords. Alternatively or additionally, the tap system 640 may beconstructed so that either one of two plugs may be coupled to the firstnetwork 602 and the first device 612. For example, the tap system 640may sense (e.g. by detecting phone line power) which end is plugged intothe first network 602 and which end is plugged into the first device 612and configure itself accordingly.

As another example, the first device 612 may be a different electronicdevice that may be converted to device similar to the first device 112of FIG. 1 via a software update or adding an application to the firstdevice 612.

FIG. 7 illustrates an example environment 700 for transcription ofcommunications. The environment 700 may be arranged in accordance withat least one embodiment described in the present disclosure. Theenvironment 700 may include a first network 702, a second network 704, athird network 706, a remote device 710, a first device 712, a seconddevice 714, a third device 716, a transcription system 730, and a tapsystem 740.

In some embodiments, the first network 702, the second network 704, thethird network 706, the remote device 710, the first device 712, thesecond device 714, and the transcription system 730, may be analogous tothe first network 202, the second network 204, the third network 206,the remote device 210, the first device 212, the second device 214, andthe transcription system 230, respectively, of FIG. 2 . Accordingly, nofurther explanation is provided with respect thereto. In these and otherembodiments, the tap system 740 may be analogous to the tap system 640of FIG. 6 . Accordingly, no further explanation is provided with respectthereto.

The third device 716 may be any electronic or digital computing device.For example, third device 716 may include a desktop computer, a laptopcomputer, a smartphone, a mobile phone, a tablet computer, or any othercomputing device that may be used to present transcriptions of audio ofa communication session between users of the remote device 710 and thefirst device 712.

An example operation of the environment 700 follows. The remote device710 may establish a communication session over the first network 702with the first device 712. The audio of the communication session may berouted through the tap system 740. The tap system 740 may split theaudio and send the audio to the first device 712 and through the secondnetwork 704 to the second device 714. The second device 714 may send theaudio to the transcription system 730 over the third network 706. Thetranscription system 730 may generate a transcription of the audio andsend the transcription to the second device 714 over the third network706. The second device 714 may send the transcription to the thirddevice 716 over the second network 704. The third device 716 may presentthe transcription of the audio. The third device 716 may present thetranscription of the audio in substantially real-time such that theaudio and the transcription of the audio are presented substantiallysynchronized.

In some embodiments, the transcription may be presented by anapplication running on the third device 716. In these and otherembodiments, a user of the third device 716 and the first device 712 mayhave the option of opening the application when the communicationsession begins to view the transcriptions. Alternately or additionally,the transcription system 730 may obtain a message from the tap system740 regarding the start of the communication session. In these and otherembodiments, the transcription system 730 may direct a message to thethird device 716 to open the application so that transcriptions maystart displaying automatically. Additionally, or alternatively, theapplication may present a visual or audible alert to indicate thattranscriptions are available. If the user responds affirmatively with anaudible command, screen click, button press, or using other input modes,then presentation of the transcriptions may begin.

In some embodiments, the third device 716 may be configured to presentthe transcriptions based on a location of the third device 716 withrespect to the first device 712. Alternately or additionally, thetranscription system 730 may be configured to generate thetranscriptions based on the location and/or configuration of the thirddevice 716. For example, if the third device 716 is not close enough tothe first device 712 for the user to see the transcriptions on the thirddevice 716, the transcriptions may not be presented by the third device716 or the transcription system 730 may not generate the transcriptions.Alternately or additionally, if the third device 716 is inactive orinaccessible, the transcription system 730 may not generate thetranscriptions.

In these and other embodiments, the generation of the transcriptions bythe transcription system 730 may be dynamic during the communicationsession such that the transcription system 730 may monitor the thirddevice 716. In response to a change in the configuration or location ofthe third device 716, the transcription system 730 may start or stoptranscription of the audio and/or sending the transcription of theaudio, and/or the transcription system 730 may select between differenttranscription systems to generate the transcription of the audio.

For example, the transcription system 730 may stop generatingtranscriptions of audio in response to the distance between the thirddevice 716 and the first device 712 dropping below a selected thresholdor until the third device 716 is available to present thetranscriptions. For example, if the user is on a phone call on the firstdevice 712 and the third device 716 is in another room (relative to thefirst device 712) and/or is turned off, transcriptions may be generated.In these and other embodiments, the first device 712 may include acordless handset, hearing aid, hearing loop, BLUETOOTH® device, or otherseparable speaking/listening device connected via a wired or wirelessconnection. In instances in which the first device 712 includes multipleparts, such as a base station and cordless handset, the determinationthat the devices are close enough may be based on how close one of themultiple parts (e.g., the cordless handset) is to the third device 716.Alternately or additionally, a microphone on the tap system 740 or onthe third device 716 may collect ambient sound from the nearby area. Thetap system 740 may compare the collected ambient sound to audiocollected from the first device 712 and to determine whether the user ofthe first device 712 is in proximity to the ambient microphone and usethe determination to turn transcriptions on and off. For example, if theambient sound is spectrally similar to sound from the first device 712or if a signal from the ambient microphone is detected at the same timeas a signal from the first device 712, the tap system 740 ortranscription system 730 may conclude that the user is likely to be invisual range of a display and turn transcriptions on.

As another example, in response to the configuration and location of thethird device 716 indicating to the transcription system 730 to generatetranscriptions, the transcription system 730 may generate transcriptionsusing a first transcription technique that may result in higher accuracytranscriptions on average than other transcription techniques. The firsttranscription technique may include re-voicing of audio, the combinationof re-voicing of audio and automatic transcription, or the combinationof multiple automatic transcriptions, or other techniques that mayresult in higher accuracy transcriptions on average than othertranscription techniques as described in U.S. patent application Ser.No. 16/209,623 filed on Dec. 4, 2018 and entitled “TRANSCRIPTIONGENERATION FROM MULTIPLE SPEECH RECOGNITION SYSTEMS,” the entirety ofwhich is incorporated herein by reference.

The other transcription techniques may be used in response to one orboth of the configuration and location of the third device 716indicating to the transcription system 730 to not generatetranscriptions or to generate transcriptions that are not forpresentation by the third device 716 in real-time with the presentationof the audio by the first device 712. In these and other embodiments,the other transcription technique may include a single automatictranscription system being used to generate the transcriptions or othertranscription techniques that may result in lower accuracytranscriptions on average than the techniques used when thetranscription system 730 is generating transcriptions for presentationby the third device 716 in real-time with the presentation of the audioby the first device 712.

Modifications, additions, or omissions may be made to the environment700 and/or the components operating in the environment 700 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the environment 700 may not include the tap system 740. Insome embodiments, the tap system 740 may be implemented as two or moreseparate devices, one connected to the second network 704 and anotherconnected to the first device 712. In these and other embodiments, thetwo or more separate devices may send audio and/or data to each othervia a wireless communication channel. In these and other embodiments,the communication session may be a voice over internet protocol (VOIP)communication session that may be routed through the second device 714.The second device 714 may provide the audio to the first device 712 overthe second network 704 and to the transcription system 730 over thethird network 706 and/or the first network 702.

As another example, in some embodiments, the environment 700 may notinclude the connection between the tap system 740 and the second network704 and the transcription system 730 may be coupled to the first network702. In these and other embodiments, the tap system 740 may not captureaudio or receive transcriptions but may rather serve as a dialing andcall forwarding server as described with respect to FIGS. 9A and 9B.

In some embodiments, the tap system 740 may be controlled from thetranscription system 730 via a communication session therebetween. Inthese and other embodiments, the tap system 740 may block the firstdevice 712 from ringing when a communication request is obtained fromthe transcription system 730 and communicates via DTMF. The tap system740 may activate call forwarding and redirect outbound calls asinstructed by the transcription system 730. Alternately or additionally,the transcription system 730 may communicate with the tap system 740during regular voice calls via data channels hiding on the voice linesuch as described with respect to FIGS. 10-13 .

FIG. 8 illustrates an example environment 800 for user monitoring thatincorporates an environment for transcription of communications. Theenvironment 800 may be arranged in accordance with at least oneembodiment described in the present disclosure. The environment 800 mayinclude a first network 802, a second network 804, a remote device 810,a first device 812 a, a second device 812 b, a third device 812 c,collectively the devices 812, and a monitor system 820. In general, theenvironment 800 may operate to monitor a user of the devices 812. Themonitoring may help an individual associated with the user, such as anadult child of the user to know a status of the user (e.g., health,mental state, location).

The first network 802 may be configured to communicatively couple theremote device 810 and the monitor system 820. The first network 802 andthe remote device 810 may be analogous to the network 102 and remotedevice 110 of FIG. 1 respectively, and thus no further description isprovided with reference to FIG. 8 .

The second network 804 may be configured to communicatively couple theremote device 810 with the first device 812 a, the second device 812 b,and the third device 812 c. The second network 804 may be analogous tothe network 102 of FIG. 1 , and thus no further description is providedwith reference to FIG. 8 .

The first device 812 a, the second device 812 b, and the third device812 c may be electronic devices that may provide data to the monitorsystem 820. The data may include information about a user beingmonitored. Examples of the devices 812 and the data that the devices 812may provide to the monitor system 820 is now provided.

In some embodiments, one or more of the devices 812 may be a phone, suchas a captioned phone, a landline phone, a mobile phone, a softphone, anapp-based phone, a videophone, a VOIP phone, among other types ofphones. The phones may provide usage data, including current and pastusage data to the monitor system 820. In these and other embodiments,when the phone is a captioned phone, the captioned phone may providetranscriptions of audio of communication sessions to the monitor system820.

In some embodiments, one or more of the devices 812 may be a locationmonitor that monitors the location of the user. The location monitor maybe a separate device or integrated into a device such as a smartphone(or other mobile device), a wearable device such as a medical alertsensor or a watch, or another position tracker carried by thesubscriber. The location of the user may be determined by the globalpositioning system (GPS), A-GPS (a.k.a. Assisted GPS using cell towerdata), near wireless location tracking (e.g. a wireless positioningsystem that locates a device using wireless signals), an indication ofwhich wireless network the devices 812 are in proximity to or loggedinto, among others. The location may be provided to the monitor system820.

In some embodiments, one or more of the devices 812 may be a navigationssystem in a smartphone, personal computer, or vehicle. The data mayinclude the GPS destination or a programmed route that may be providedto the monitor system 820.

In some embodiments, one or more of the devices 812 may be a medicalalert sensor carried by the user. The medical alert sensor may beactivated by fall detection, breathing/heartbeat sensors, motion, lackof motion, pressing a button, placing a call, etc. The medical alertsensor may provide an alert or other data to the monitor system 820.

In some embodiments, one or more of the devices 812 may be a motionsensor, infrared sensor, switch, temperature sensor, sensor triggeredwhen a light beam is broken, pressure sensor in places such as the bed,chairs, and floor mats, and other sensors. The sensors may detectmotion, opening and closing doors (including garage doors) and windows,and motion of people at one or more locations. The sensors may providethe sensed data to the monitor system 820.

In some embodiments, one or more of the devices 812 may be a microphone.The microphone may be part of another device which may be part of thephone, computing device, home appliance such as a smart speaker, orseparate. The microphone may provide sound data to the monitor system820.

In some embodiments, one or more of the devices 812 may be a camera. Thecamera may be part of another device which may be part of the phone,computing device, home appliance such as a smart speaker, or separate.The camera may provide image data to the monitor system 820.

In some embodiments, one or more of the devices 812 may be homeappliances or other home devices such as a television, refrigerator,oven, microwave, HVAC controls, room lights, desk lights, floor lights,smoke, fire, carbon monoxide, or other emergency detectors, thermalsensors, motion detectors, intrusion detectors such as door or windowsensors, etc. The home appliances or home device may send usage dataand/or alerts to the monitor system 820.

In some embodiments, one or more of the devices 812 may be a smartspeaker. The smart speaker may provide data to the monitor system 820such as usage, a history, (including times or) specific requests fromthe user, an audio signal the monitor system 820 may analyze, alarms(e.g. wakeup alarms) and reminders set by the user, and status andoperation of remote devices linked to the smart speaker such as remotepower switches, thermostats, etc.

In some embodiments, one or more of the devices 812 may be a link to amedical alert center. The monitor system 820 may both (a) receive userstatus information for use in providing information to others and (b)send user status information to the medical alert center.

In some embodiments, one or more of the devices 812 may be onlineinformation sources such as weather, emergency conditions, messages fromfamily, friends, and other contacts, grocery delivery services,appointment and subscription and other reminders, and notices frommedical providers and other businesses. The data from these sources maybe provided to the monitor system 820.

In some embodiments, one or more of the devices 812 may be smartmedication dispensers that provide reminders to take medication andsense when medication has been taken. The monitor system 820 may obtainusage data from the smart medication dispensers.

In some embodiments, one or more of the devices 812 may be patientmonitoring equipment such as heart and respiratory monitors, bloodglucose testers and monitors, and blood oxygen sensors. The monitorsystem 820 may obtain data from the patient monitoring equipment.

In some embodiments, one or more of the devices 812 may be a vehicle.Data provided to the monitor system 820 may include when the vehicle isor has been running, opening and closing doors, current vehicle locationand travel history, use of accessories such as radio and climatecontrol, interior lights, locked/unlocked status, proximity of awireless key, and presence of a driver and passengers, including thelocation of each, as determined, for example, by seat pressure sensors.

The monitor system 820 may be configured to monitor a user of themonitor system 820. The monitor system 820 may monitor the user based ondata collected by the devices 812. Based on the data, the monitor system820 may issue one or more alerts based on rules associated with thedata. Alternately or additionally, the monitor system 820 may determinealerts using a classifier or estimator using, for example, linear orlogistic regression, one or more neural networks, or another machinelearning system trained on data collected from other subjects anddesigned to combine one or more sources of information to make anestimate of the user's status and determine a course of action.

The alerts may be sent to the remote device 810 and/or other devices. Insome embodiments, the destination for an alert may be based on the typeof the alert. For example, in response to a first set of data associatedwith first activities of the user (e.g. routine phone calls,characteristic movement throughout the home) an alert may be sent towebpage that may include a user interface. In response to a second setof data associated with second activities of the user, an alert may besent to the remote device 810.

An example of the operation of the environment 800 follows. In someembodiments, a user may access an interface on the monitor system 820 oron a website to authorize an individual to have access to alerts fromthe monitor system 820. In these and other embodiments, the interfacemay require the user to provide a name, account number, PIN password,biometric reading such as a voice sample to be compared to a voiceprintor an image to be compared to an entry in the user's profile using faceidentification, or other identification confirmation before grantingaccess. The monitor system 820 may allow a user to determine anexpiration date for access and/or revoke access using steps (e.g.website access, identity confirmation), similar to those of theauthorization process.

In response to obtaining the authorization, the monitor system 820 maysend the credentials (e.g. username and password) to the remote device810 that is associated with the individual. Alternately or additionally,credentials may be provided to the user, who, in turn, may pass thecredentials to the individual. Alternately or additionally, the user mayselect credentials and provide them to the monitor system 820. In theseand other embodiments, the individual may use the credentials to loginto a website or other portal and observe the data regarding the user,including any alerts issued by the monitor system 820.

In some embodiments, multiple individuals may be authorized to obtaindata or alerts. In these and other embodiments, each individual may haveaccess and a separate profile for setting up a different set of alertsand criteria. In these and other embodiments, each individual may have adifferent level or the same level of access (e.g. restrictions oninformation accessed, configuration settings, and alerts received).

In some embodiments, one or more alerts may be established by default,by selection, or based on a configuration of the individual. In theseand other embodiments, in response to criteria for an alert beingsatisfied, the individual may receive an alert via a phone call, textmessage, voicemail, email, or by other means at the remote device 810.For example, the alert may activate an application on a smartphone,watch, smart speaker (e.g. Alexa), or another device that notifies theindividual of the alert and/or delivers the alert. Various types of datamay result in a determination to make an alert. Various examples follow:

An alert may be triggered in response to the user's location, asdetermined, for example, by the location of a device, a door opening, ormovement of a vehicle. For example, an alert may be triggered if theuser leaves home or crosses a specified geographical boundary within aspecified range of time or at a time inconsistent with typical behavior.

An alert may be triggered in response to the user placing a phone callto an emergency number or a medical provider; the user failing toestablish a communication session during a specified period of time suchas a selected portion of a day, a specific number of days or hourspassing since the previous communication session; or the user fails toanswer a selected number of incoming communication sessions during aspecified period of time.

An alert may be triggered in response to activity monitoring in the homeof the user. For example, activity or lack of activity such as doorsopening/closing, lights being turned off or on, and use of a car,computer, smartphone, TV, or appliances such as a microwave, HVAC, orrefrigerator.

An alert may be triggered in response to noise or lack thereof, such asambient noise such as a person walking, conducting typical activities,or speaking. In these and other embodiments, when it is determined thatone or more activity metrics fall outside a selected range or that,taken together, activity levels or patterns indicate that the user'sactivity is outside normal ranges or that there is an event thatrequires attention, an alert may be triggered. In these and otherembodiments, the noise may be monitored for speech including keywords orphrases such as “help,” “call a doctor,” a person's name, etc. Themonitor may also listen for non-speech sounds such as alarms,explosions, falling objects, fire, emergency vehicle sirens, vocalexclamations such as shouting, etc.

An alert may be triggered in response to images, such as images from acamera of infrared detector that may monitor the area for motion or forspecific activities. The images may be analyzed for motion, lights, thesubscriber's face, unfamiliar faces, etc.

An alert may be triggered in response to voice analysis of the user todetect stroke, measure cognitive decline, detect early indicators forcognitive disease such as Alzheimer's, or flag other potential medicalconditions. The voice analysis may use text patterns (e.g. changes inpatterns of words and phrases used by the subscriber, increased use offiller words such as “um,” frequency of using selected key phrases suchas “I don't remember,” “oh,” or “what?” etc.), voice signal quality(e.g. increased shakiness of pitch, reduced volume, slower speakingrate, reduced clarity of articulation, changes in the frequencyspectrum, etc.), or speaking style (e.g. length of pauses, slowerresponse after the other party stops speaking, etc.) to detect thedifferent conditions. Alternately or additionally, time of day (e.g.,compensating for possible fatigue at bedtime, for example) and patternsand statistics from previous calls with the subscriber may be used as abaseline for detecting changes or impaired abilities. Voice samples maybe obtained from ambient conversation via a microphone or from a phonesuch as a captioned phone.

An alert may be triggered in response to analysis of audio and/or atranscription of the audio of a communication session of the user. Forexample, the text analysis of the transcription may determine, forexample, that the user is stressed, is discussing a situation requiringattention, has a certain medical condition, diagnosis, or set ofsymptoms, or that the user is possibly being swindled by the other partyin the communication session.

In response to an alert, the monitor system 820 may send a text or diala number of the remote device 810. In response to a number that is notanswered, the monitor system 820 may dial an alternate number or leave avoicemail message using recorded speech or text-to-speech thatidentifies the user and specifies the type and details of the alert.Alternately or additionally, a message may be sent to an application onthe remote device 810. In these and other embodiments, the message mayprovide information regarding the alert, including the data thatresulted in the alert.

Alternately or additionally, in response to an alert, the monitor system820 may establish a communication session with another device associatedwith the user, such as one of the devices 812. In these and otherembodiments, a system, such as an interactive voice response system, mayask the user questions to be answered and then determine whether thesubscriber meets predetermined criteria for a triggering action such asnotifying the individual.

Alternately or additionally, in response to an alert, the monitor system820 may establish a communication session with an alert response servicesuch as emergency response individuals, a medical practitioner, a remotedoctor service (e.g. Teladoc), a security service, or a local hospital.A voice or video call between the individual and a second party may beestablished by the monitor system 820.

In some embodiments, an alert may include images, audio, text, aconfidence level of the alert (e.g. “possible,” “likely,” or “confirmed”or “green” “yellow,” or “red”), alert severity, and other informationconcerning the event or events that triggered the alert. Alternately oradditionally, an individual may obtain additional information regardingan alert. Additional information regarding the alert may be accessedthrough portal that provides more information such as live video and/oraudio, records of use such as phone call history (including people thesubscriber talked to by number and name, call topic as identified bynatural language processing topic detection, and time/date/duration),and results of attempts to contact an individual. In these and otherembodiments, the portal may allow the individual to attempt to contactthe user via: phone calls, text, live audio and/or video, and anintercom mode that does not require the individual to pick up thehandset or otherwise answer the call before connecting the two parties.

Modifications, additions, or omissions may be made to the environment800 and/or the components operating in the environment 800 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the monitor system 820 may include the capabilities of thefirst device 112 of FIG. 1 , the first device 212 of FIG. 2 , and otherdevices described in this disclosure that operate in a manner analogousto the first device 112 of FIG. 1 and the first device 212 of FIG. 2 .In these and other embodiments, the environment 800 may further includea transcription system that may be configured to generate transcriptionsof audio obtained by the monitor system 820 during the communicationsession. The monitor system 820 in these and other embodiments, maypresent the transcriptions to, for example, a user of the monitor system820 or to an authorized individual. As another example, in someembodiments, one of the devices 812 may be configured as the firstdevice 112 of FIG. 1 or the first device 212 of FIG. 2 , and otherdevices described in this disclosure that operate in a manner analogousto the first device 112 of FIG. 1 and the first device 212 of FIG. 2 .In these and other embodiments, the one of the devices 812 may sendaudio to a transcription system and obtain transcriptions from thetranscription system for presentation.

As another example, portions of the monitor system may be performed by aserver or a server system. For example, the server may analyze the datato determine a status of a user. In these and other embodiments, theserver may be part of a system that also includes a transcriptionsystem. In these and other embodiments, the monitor system 820 mayinclude a device that operates as the first device 112 of FIG. 1 or thefirst device 212 of FIG. 2 , and other devices described in thisdisclosure that operate in a manner analogous to the first device 112 ofFIG. 1 and the first device 212 of FIG. 2 and that collects data fromthe devices 812. In these and other embodiments, the device of themonitor system 820 may send the data to the server.

FIG. 9A illustrates an example environment 900 for routing audio and atranscription associated with a communication session. The environment900 may be arranged in accordance with at least one embodiment describedin the present disclosure. The environment 900 may include a firstnetwork 902, a second network 904, a remote device 910, a transcriptionsystem 930, and a presentation system 906.

In some embodiments, the first network 902 may be analogous to thenetwork 102 of FIG. 1 . In the illustrated example of FIG. 9A, the firstnetwork 902 may be configured to communicatively couple an audio system912 of the presentation system 906, the remote device 910, and/or thetranscription system 930. Additionally or alternatively, the firstnetwork 902 may be configured to communicate audio between the audiosystem 912, the transcription system 930, and the remote device 910.

The second network 904 may be analogous to the third network 206 of FIG.2 . In the illustrated example of FIG. 9A, the second network 904 may beconfigured to communicatively couple the presentation system 906 withthe transcription system 930. Additionally or alternatively, the secondnetwork 904 may be configured to communicate a transcription of thecommunication session from the transcription system 930 to thetranscription presentation system 914. Although illustrated as separatenetworks, in some embodiments, the second network 904 may be part of thefirst network 902.

In some embodiments, the remote device 910 may be analogous to theremote device 110 of FIG. 1 . The presentation system 906 may includeany suitable system or device that may be used for communication betweenusers of the presentation system 906 and the remote device 910.

In some embodiments, the presentation system 906 may include an audiosystem 912. The audio system 912 may include any suitable system ordevice configured to communicate and/or receive audio during acommunication session between the presentation system 906 and the remotedevice 910. Additionally or alternatively, the audio system 912 mayinclude any suitable system or device configured to present receivedaudio and/or generate audio based on sound (e.g., speech) obtainedduring the communication session. For example, the audio system 912 mayinclude a microphone and/or a speaker. In these or other embodiments,the audio system 912 may include one or more digital and/or analogcomponents that are configured to receive and/or communicate audio overthe first network 902.

In some embodiments, the presentation system 906 may include alsoinclude a transcription presentation system 914. The transcriptionpresentation system 914 may include any suitable system or deviceconfigured to receive and/or present the transcription of thecommunication session. For example, the transcription presentationsystem 914 may include one or more digital and/or analog components thatare configured to receive data (e.g., transcriptions) over the secondnetwork 904. Additionally or alternatively, the transcriptionpresentation system 914 may include any suitable system or deviceconfigured to present the received transcription. For example, thetranscription presentation system 914 may include any suitable displaydevice such as a television, a computer monitor, a telephone screen,etc. As indicated above, in the example embodiment of FIG. 9A, thetranscription presentation system 914 may be communicatively coupled tothe transcription system 930 via the second network 904 such that thetranscription may be communicated from the transcription system 930 tothe transcription presentation system 914 via the second network 904.

In some embodiments, the presentation system 906 may include a userinterface 916, which may be any suitable system or device configured toreceive user input related to establishing a communication session. Forexample the user interface may include a dial pad and associatedcomponents that are configured to receive a telephone number as aninput. The dial pad may be a physical dial pad, a virtual dial padpresented on a touchscreen, or any suitable combination thereof.

In some embodiments, the audio system 912, the transcriptionpresentation system 914, and the user interface 916 of the presentationsystem 906 may be integrated into a single device such as the firstdevice 112 of FIG. 1 such that the presentation system 906 may be thedevice. Additionally or alternatively, one or more of the audio system912, the transcription presentation system 914, and the user interface916 may be included in separate devices that are communicatively coupledvia wired and/or wireless connections. In these or other embodiments,the presentation system 906 may include any number of devices that mayfacilitate conducting communication sessions and presentingcorresponding transcriptions. For example, the presentation system 906may include one or more landline phones, cellular phones, smartphones,personal computers, routers, tap systems (such as described with respectto FIGS. 6 and 7 ), tablet computers, the first device 112 of FIG. 1 ,the first device 212 of FIG. 2 , the second device 214 of FIG. 2 , thesecond network 204 of FIG. 2 , or any suitable combination thereof.

An example of the operation of the environment 900 is now provided. Insome embodiments, the communication session may be established betweenthe presentation system 906 and the remote device 910. In these or otherembodiments, the communication session may be an audio communicationsession such as a telephone call. In some embodiments, the communicationsession may be established such that audio that originates at the remotedevice 910 and that is received at the audio system 912 is routed to orthrough the transcription system 930.

In some instances, the communication session may be established inresponse to being initiated at the presentation system 906. In otherinstances, the communication session may be established in response tobeing initiated at the remote device 910. In some embodiments, theestablishment of the routing of audio to or through the transcriptionsystem may be based on whether the communication session is initiated atthe presentation system 906 or at the remote device 910. In the presentdisclosure, the initiation of the communication session may be describedfrom the perspective of the presentation system 906. For example,initiation of the communication session at the presentation system 906may be referred to as an “outbound call”. As another example, initiationof the communication session at the remote device 910 may be referred toas an “inbound call”.

In some embodiments, for outbound calls, a user of the presentationsystem 906 may begin initiation of the communication session with theremote device 910. For example, the presentation system 906 may includea telephone and the user may remove the telephone from the hook to begininitiation of the communication session. As another example, thepresentation system 906 may include a smartphone and the user may usethe built-in telephone features of the smartphone or open an application(also referred to as an “app”) configured to establish communicationsessions to begin initiation of the communication session.

In some embodiments, the presentation system 906 may detect theinitiation of the communication session and, in response to detectingthe initiation of the communication session, may establish a first audioconnection 940 with the transcription system 930 via the first network902. Initiation of the communication session may be in response to, forexample, a user starting to dial one or more digits, opening an app orsoftware, going off-hook (e.g., for a landline phone), pressing “Send”on a mobile phone, clicking on a screen icon, issuing a voice command,invoking speed dialing, etc. For example, the presentation system 906may dial a telephone number that is associated with the transcriptionsystem 930 (referred to as a “transcription system number”) to establishthe first audio connection 940. The first audio connection 940 may beany suitable analog and/or digital connection that may be used tocommunicate audio. In the example of FIG. 9A, the arrows and linesillustrated as representing the first audio connection 940 are merely tohelp with visualizing that the first audio connection 940 is between thepresentation system 906 and the transcription system 930. The arrows andlines are not meant to represent the actual path of the first audioconnection 940. For example, as indicated above, the path of the firstaudio connection 940 may be through the first network 902, even thoughthe lines and arrows that represent the first audio connection 940 arenot illustrated inside of the first network 902.

By way of example, in some instances, the first audio connection 940 maybe established over a POTS line and DTMF-based messaging may be used toestablish the first audio connection 940. Additionally or alternatively,the first audio connection 940 may include a session initiation protocol(SIP) connection (as with a VoIP device) or other non-POTS line. Inthese or other embodiments, the step of the presentation system 906dialing the transcription system number may be replaced with othersignaling. For example, for phone types other than POTS phones, theDTMF-based messaging may be replaced by corresponding signalsappropriate to the technology. For example, if SIP messages are used inplace of DTMF, the presentation system 906 may send a SIP connect,transfer, or conference request to the transcription system 930 or to anappropriate entity that is part of the first network 902.

In some embodiments, the presentation system 906 may establish the firstaudio connection 940 as soon as the initiation of the communicationsession is detected. Additionally or alternatively, the presentationsystem 906 may establish the first audio connection 940 while the useris providing input related to establishing the communication sessionwith the remote device 910. For example, the presentation system 906 mayestablish the first audio connection 940 while the user is entering,e.g., via the user interface 916, a telephone number or other deviceidentifier that is linked to the remote device 910 (referred to as a“remote device number”).

In some embodiments, establishment of the first audio connection 940 maybe via a “phone line” that corresponds to a telephone number associatedwith the presentation system 906 (referred to as a “presentation systemnumber”). In the present disclosure, a “phone line” may refer to aphysical landline telephone line that corresponds to a particulartelephone number. Additionally or alternatively, a “phone line” mayrefer to a mobile phone account that is assigned a particular telephonenumber. In addition, reference to a particular telephone number beinglinked to a particular device or system (e.g., the presentation system906 or the remote device 910) may refer to the particular device orsystem being configured to conduct communication sessions (e.g., placetelephone calls, receive telephone calls, participate in telephonecalls) using the particular telephone number.

For example, a particular system or device may have a physical phonewire of a particular landline phone line plugged into it. As such, theparticular system or device may be configured to conduct communicationsessions using the particular telephone number that corresponds to theparticular landline phone line. As another example, the particularsystem or device may have a subscriber identification module (SIM) cardinstalled therein. The SIM card may correspond to a particular telephonenumber and may enable the particular system or device to conductcommunication sessions using the particular telephone number. As anotherexample, a first telephone number may be associated with the particularsystem or device. Further, communication sessions associated with asecond telephone number may be routed through the first telephone number(e.g., via call forwarding). The particular system or device may thus beconfigured to conduct communication sessions using the second telephonenumber through the routing through the first telephone number and theassociation of the particular system or device with the first telephonenumber.

In some embodiments, the presentation system 906 may include a numberbuffer that may store at least part of the digit sequence of the remotedevice number to the remote device 910 as received from the user. Assuch, in instances in which the user may begin entering the remotedevice number prior to the first audio connection 940 being establishedand/or prior to the transcription system 930 being ready to receive theremote device number or indicating it is ready to receive the remotedevice number, the already entered digits may not be lost. Additionallyor alternatively, the number buffer may allow for the process ofestablishing the first audio connection 940 and dialing of at least partof the remote device number to happen at the same time, which may reducedelay that may be perceived by the user.

In these or other embodiments, the presentation system 906 may beconfigured to reduce user perception of the time taken with respect toestablishing the first audio connection 940. For example, while thepresentation system 906 is dialing the transcription system number asaudio tones, the presentation system 906 may be configured to mute anearpiece of the audio system 912 such that the user does not hear theaudio tones. Additionally or alternatively, the muting may continue suchthat the user does not hear ring tones related to establishing the firstaudio connection 940. In these or other embodiments, the presentationsystem 906 may be configured to mute the dialed audio tones and/or thering tones and also generate an artificial dial tone or other indicationthat the presentation system 906 is ready to accept a destination numbersuch as the remote device number. Additionally or alternatively, whilethe first audio connection 940 is being established, the presentationsystem 906 may be configured to generate a dial tone, ringing tones, orany other call progress indicator that may convey to the user that acommunication session with the remote device 910 has been initiated.

In these or other embodiments, to reduce delay and/or cost, thepresentation system 906 may be configured to identify which center ofthe transcription system 930 may be within a particular geographicdistance of the presentation system 906. In some embodiments, thepresentation system 906 may be configured to identify which center isgeographically closest to the location of the presentation system 906.In these or other embodiments, the presentation system 906 may beconfigured to establish the first audio connection 940 with a particularcenter that is within the particular geographic distance or that isclosest to the location of the presentation system 906. In someembodiments, the presentation system 906 may be configured to determinewhich center is within the particular geographic distance and/or closestbased on area codes of the telephone numbers associated with the centersand the area code of the presentation system number.

In some embodiments, the presentation system 906 may have previouslyestablished the first audio connection 940 prior to detecting initiationof the communication session. In such instances, the presentation system906 may be configured to maintain the first audio connection 940 untilinitiation of the communication session.

In some embodiments, in response to the first audio connection 940 beingestablished, the transcription system 930 may communicate a confirmationsignal (e.g., a click, tone, or other audio indicator or an out-of-bandsignal such as a SIP message) to the presentation system 906 that mayindicate that the first audio connection 940 has been established. Theconfirmation signal may also indicate that a second audio connection 942may be established between the remote device 910 and the transcriptionsystem 930. The second audio connection 942 may be any suitable digitalor analog audio connection such as described with respect to the firstaudio connection 940. In the example of FIG. 9A, the arrows and linesillustrated as representing the second audio connection 942 are merelyto help with visualizing that the second audio connection 942 is betweenthe remote device 910 and the transcription system 930. The arrows andlines are not meant to represent the actual path of the second audioconnection 942. For example, the path of the second audio connection 942may be through the first network 902 and/or the presentation system 906,even though the lines and arrows that represent the second audioconnection 942 are not illustrated as such.

In some embodiments, the transcription system 930 may be configured toestablish the second audio connection 942. For example, in someembodiments, the communication of the confirmation signal to thepresentation system 906 may indicate to the presentation system 906 thatthe transcription system 930 is ready to receive user input related toestablishing the second audio connection 942 (e.g., the telephone numberthat is linked to the remote device 910). As such, in response toreceiving the confirmation signal, the presentation system 906 maycommunicate the user input to the transcription system 930 via the firstaudio connection 940. For instance, the presentation system 906 maycommunicate to the transcription system 930 the destination numberentered by the user at the presentation system 906.

In some embodiments, the transcription system 930 (e.g., using anInteractive Voice Response System (“IVR”)) may establish the secondaudio connection 942 with the remote device 910 using the receivedremote device number. For example, the transcription system 930 may dialthe remote device number using any appropriate signaling such asDTMF-based signaling, SIP signaling, etc. In some embodiments, to reduceand/or minimize delay, the transcription system 930 may be configured tobegin dialing the remote device number while the user is still enteringdigits of the remote device number.

As an example, an outbound call may start when the user initiates a callby going off-hook or starting to dial a phone number for a remote device910, which may trigger setting up the first audio connection 940. Oncethe user has provided the phone number or once the first audioconnection 940 is set up, the number may be sent to the transcriptionsystem 930, where the phone number may be used to connect to the remotedevice 910 via the second audio connection 942 Once the presentationsystem 906 is connected to the remote device 910 and a conversationbegins, the transcription system 930 converts audio signals passingthrough it to text and forwards the text to the presentation system 906to be displayed as transcriptions.

In some embodiments, the transcription system 930 may be configured tolink the first audio connection 940 and the second audio connection 942in a manner that establishes a third audio connection 944 between thepresentation system 906 and the remote device 910. The third audioconnection 944 may be any suitable digital or analog audio connectionsuch as described with respect to the first audio connection 940. In theexample of FIG. 9A, the arrows and lines illustrated as representing thethird audio connection 944 are merely to help with visualizing that thesecond audio connection 942 is between the remote device 910 and thepresentation system 906. The arrows and lines are not meant to representthe actual path of the third audio connection 944. For example, asindicated below, the path of the third audio connection 944 may bethrough the first network 902 and/or through the transcription system930, even though the lines and arrows that represent the third audioconnection 944 are not illustrated as such.

The third audio connection 944 may be established for conducting thecommunication session. In addition, the linking of the first audioconnection 940 and the second audio connection 942 may be such that thetranscription system 930 receives first audio that originates at theremote device 910 during the communication session. Additionally oralternatively, the linking of the first audio connection 940 and thesecond audio connection 942 may be such that the transcription system930 receives second audio that originates at the presentation system 906during the communication session.

For example, in some embodiments, the transcription system 930 mayestablish the third audio connection 944 by establishing or initiating athree-way call between the presentation system 906, the remote device910, and the transcription system 930. In some embodiments, thetranscription system 930 may be configured to host (e.g., establish,manage, and maintain) the three-way call. Additionally or alternatively,a service provider may host the three-way call in response to anindication received from the transcription system 930.

Additionally or alternatively, the transcription system 930 mayestablish the third audio connection 944 by acting as an intermediarybetween the first and second audio connections. For example, thetranscription system 930 may receive, via the second audio connection942, the first audio and may relay the first audio to the presentationsystem 906 via the first audio connection 940. Similarly, thetranscription system 930 may receive the second audio via the firstaudio connection 940 and relay the first audio to the remote device 910via the second audio connection 942.

In some embodiments, the presentation system 906 may be configured toestablish the second audio connection 942 and/or the third audioconnection 944 instead of the transcription system 930. For example, insome embodiments, the communication of the confirmation signal to thepresentation system 906 may indicate to the presentation system 906 thatthe first audio connection 940 has been established. In these or otherembodiments, in response to receiving the confirmation signal, thepresentation system 906 may establish the second audio connection 942and the third audio connection 944. For example, the presentation system906 may establish or initiate a three-way call between the presentationsystem 906, the remote device 910, and the transcription system 930. Insome embodiments, the presentation system 906 may be configured to hostthe three-way call. Additionally or alternatively, a service providermay establish, manage, and maintain the three-way call in response to anindication received from the presentation system 906.

In these or other embodiments, the presentation system 906 may beconfigured to establish the third audio connection 944 between thepresentation system 906 and the remote device 910 and may establish thesecond audio connection 942 by relaying audio received over the firstaudio connection 940 and the third audio connection 944. For example,the presentation system 906 may dial the remote device number via asecond landline phone line to establish the third audio connection 944.In these or other embodiments, the presentation system 906 may establishthe second audio connection 942 by bridging the first audio connection940 and the third audio connection 944.

For example, the presentation system 906 may receive, via the thirdaudio connection 944, the first audio that originates at the remotedevice 910 and may relay the first audio to the transcription system 930via the first audio connection 940. In these or other embodiments, thepresentation system 906 may communicate the second audio that originatesat the presentation system 906 to the transcription system 930.Additionally or alternatively, the bridging may include establishing athree-way call.

For inbound calls, the establishment of the communication sessionbetween the remote device 910 and the presentation system 906 for therouting of the audio to or through the transcription system 930 mayoccur through various mechanisms. For example, in some embodiments, aparticular telephone number may be assigned to the presentation system906 and linked to the transcription system 930. The linking of theparticular telephone number to the transcription system 930 may be suchthat the second audio connection 942 between the transcription system930 and the remote device 910 may be established in response to theremote device 910 dialing the particular telephone number. In these orother embodiments, the transcription system 930 may have stored thereonthat the particular telephone number is associated with the presentationsystem 906. Further, the transcription system 930 may then establish thefirst audio connection 940 between the presentation system 906 and thetranscription system 930 in response to the particular telephone numberbeing associated with the presentation system 906 and in response to thesecond audio connection 942 being established by the dialing of theparticular telephone number. In these or other embodiments, thetranscription system 930 or the presentation system 906 may establishthe third audio connection 944 with three way calling or relaying ofaudio, such as described above.

Another example of establishing the communication session for inboundcalls may include the remote device 910 initiating establishment of thethird audio connection 944 by dialing a presentation system number thatis linked to the presentation system 906. In these or other embodiments,the presentation system 906 may automatically answer the call and thenmay establish the first audio connection 940 and the second audioconnection 942 with three way calling or relaying of audio, such asdescribed above. In these or other embodiments, the presentation system906 may suppress ringing as described in further detail below while thefirst audio connection 940 and/or the second audio connection 942 arebeing established.

Additionally or alternatively, the presentation system 906 may transfer(e.g., using flash-hook transfer, SIP REFER, etc.) the communicationsession (e.g., transfer a call) to the transcription system 930 suchthat the second audio connection 942 may be established. An identifierof the presentation system 906 (e.g., the presentation system number, orother call identifier) may be sent as part of the transfer such that thetranscription system 930 may know where to call back. The transcriptionsystem 930 may then establish the first audio connection 940 and thethird audio connection 944.

In these or other embodiments, the presentation system 906 may suppressringing as described in further detail below while the second audioconnection 942 and/or the third audio connection 944 are beingestablished. Establishment of the first audio connection 940 and/or thethird audio connection 944 may include re-dialing the telephone numberlinked to the presentation system 906 such that the presentation system906 rings to allow for answering the call to establish the communicationsession.

Another example of establishing the communication session for inboundcalls may include the remote device 910 initiating establishment of thethird audio connection 944 by dialing the presentation system number,but with the call being forwarded to the transcription system 930. Afterthe call is forwarded to the transcription system 930, the transcriptionsystem 930 may answer the call to establish the second audio connection942. The transcription system 930 may then establish the first audioconnection 940 and/or the third audio connection using three way callingor audio relaying such as described above.

Examples of how the call forwarding may be accomplished are nowdiscussed.

In some embodiments, the presentation system 906 may communicate withthe telephone service provider of the presentation system 906 to set upthe call forwarding. In some embodiments, this action may be triggeredby an initialization of the presentation system 906 (e.g., power-up,reset, a plug-in, etc.), by an installer of the presentation system 906,by a user of the presentation system 906, by the transcription system930 (e.g., a support center of the transcription system 930), by anerror message indicating that calls are not properly forwarded or thattranscription of communication sessions has not happened for a while, bythe presentation system 906 knowing that (a) call forwarding has not yetbeen setup and that (b) the presentation system 906 is not currentlyparticipating in a communication session (e.g., a phone call), etc.

In some embodiments, call forwarding may be set up using DTMF signaling.For example, the call forwarding may be set up by the presentationsystem 906 dialing a selected number to the service provider and playinga DTMF string such as #405 #cap_number #, where “cap_number” is a numberassociated with the transcription system 930 (referred to herein as the“forwarding number”). In some instances, the presentation system 906 maynot know which service provider is the carrier for the phone numberlinked to the presentation system 906. In such instances, thepresentation system 906 may try using codes for multiple serviceproviders. The transcription system may access a record or databaselinking the forwarding number to a number associated with thepresentation system 906 so that it may, for example, connect an incomingcall to the forwarding number to the associated presentation system 906.The transcription system may detect the forwarding number of an incomingcall using a dialed number identification service (DNIS), then determinethe number of the presentation system 906 using the record or database.

In these or other embodiments, the presentation system 906 may obtainthe forwarding number (e.g., cap_number) from a customer record ordatabase linked to the presentation system 906 that may be obtained bythe presentation system 906. In these or other embodiments, thepresentation system 906 may call a service that detects Caller ID andtells the presentation system 906 its phone number (e.g. using DTMFtones). The presentation system 906 may create a record or databaseentry linking its phone number to the forwarding number or it mayprovide its phone number to the transcription system 930. Additionallyor alternatively, in instances in which the presentation system 906includes a cell phone, a corresponding cell phone option for activatingcall forwarding may be utilized (e.g. from a VERIZON® cell phone, dial*72 plus the forwarding number). In these or other embodiments, an IVRof the presentation system 906 may monitor tones and other announcementsfrom the service provider during the forwarding setup call to ensurethat call forwarding is set up correctly. Additionally or alternatively,the IVR may place a second call to query the service provider todetermine whether forwarding is set up correctly.

Additionally or alternatively, instead of the presentation system 906requesting call forwarding, a proxy such as a server in thetranscription system 930 or a Private Branch Exchange (PBX) activated bya smartphone app (which may be part of the presentation system 906) mayplace the request to forward calls and may spoof the presentation systemnumber (e.g., by spoofing the ANI (automatic number identification),CLID (calling line identification), call display, or other Caller IDservice of the presentation system 906), since the call may not comedirectly from the presentation system 906. Other methods for activatingcall forwarding such as with messages (e.g. SIP message) to/from theservice provider, using an API, or via a web site may also be used. Anysuitable complimentary process may be used to cancel call forwarding.Further details with respect to using a call forwarding server arediscussed below with respect to FIG. 9B.

If call forwarding fails or if the presentation system 906 detects apossible problem, the presentation system 906 may capture either thetone sequence or the actual audio and any log info related to theforwarding failing. In these or other embodiments, the presentationsystem 906 may then communicate with the transcription system 930 (e.g.,dial the presentation system 906 using a dial-up modem or by any othersuitable mechanism) and provide the results to a human or machine agentassociated with the transcription system 930. The agent may then sendthe presentation system 906 further instructions, such as to modify theforwarding number string entered and try again. If at any time thepresentation system 906 is in a first communication session with theservice provider or with the transcription system 930 and a userattempts to initiate a second communication session (e.g., picks up thehandset, opens a calling app, etc.) the first communication session maybe immediately disconnected. In some embodiments, once forwarding is setup, the presentation system 906 may inform the transcription system 930that call forwarding has been set up and may advise the transcriptionsystem 930 of the forwarding number and presentation system 906 number.

In some instances, the transcription services (e.g., receiving and/orpresenting of the transcription) provided at the presentation system 906may be disabled (e.g., in response to user input, changes in the user'saccount such as losing certification to receive captions, or lack ofpayment). In some embodiments, in response to the transcription servicesprovided at the presentation system 906 being disabled, the presentationsystem 906 may alert the call center not to send the transcription. Inthese or other embodiments, the next time the presentation system 906 isidle, it (or a proxy) may cancel call forwarding. In these or otherembodiments, the transcription services provided at the presentationsystem 906 may be re-enabled (e.g., in response to user input). Inresponse to the transcription services being re-enabled, the callforwarding may be set up again such as described above. Additionally oralternatively the call forwarding may always be enabled, and thecommunication or presentation of the transcription may be disabled inresponse to the transcription services being disabled.

After call forwarding has been enabled, when the remote device 910 callsthe presentation system 906, the service provider may forward the callto the transcription system 930 such that the second audio connection942 between the transcription system 930 and the remote device 910 maybe established. The transcription system 930 may then obtain thepresentation system number to be able to establish the first audioconnection 940 and the third audio connection 944.

In some embodiments, the transcription system 930 may receive the numberdialed at the remote device 910 (e.g., the presentation system number)in any suitable manner to obtain the presentation system number. Forexample, in some embodiments, the dialed number may be communicated tothe transcription system 930 using the dialed number identificationservice (DNIS).

Alternatively, the presentation system 906 may be associated with aunique forwarding number so that the transcription system 930 knows thepresentation system 906 identity (e.g., the presentation system number)based on the inbound call arriving at the unique forwarding number. Insuch instances, the presentation system number (e.g., the home telephonenumber of a user associated with the presentation system 906) may bemaintained.

In these or other embodiments, after the transcription system 930obtains the presentation system number, the transcription system 930 mayestablish the first audio connection 940 and the third audio connection944 in any suitable manner such as by establishing a three-way call orrelaying audio as described above with respect to outbound calls. Inthese or other embodiments, the transcription system 930 may detect theremote device number (e.g., via the ANI of the remote device number) andmay forward it to the presentation system 906 such that the incomingcall appears at the presentation system 906 (e.g., on caller ID) ashaving originated from the remote device 910.

In some embodiments, notification of an inbound call (e.g., ringing atthe presentation system 906) may be suppressed at the presentationsystem 906 while the call forwarding is occurring. For example, in someinstances, before an inbound call from the remote device 910 isforwarded to the transcription system 930, the presentation system 906may begin to present a notification of the inbound call (e.g., atelephone of the presentation system 906 may being ringing). However,the notification may abruptly stop once the forwarding occurs and thenmay begin again when the transcription system 930 finishes establishingthe communication session by initiating establishment of the first audioconnection 940 and the third audio connection 944 (e.g., by calling thepresentation system 906). In some embodiments, notification suppressionmay be enabled with respect to the presentation system 906 such that thenotification is not presented until the transcription system 930finishes establishing the communication session. For example, thetelephone of the presentation system 906 may not ring when receiving theinitial call from the remote device 910 but may ring when receiving thesubsequent call from the transcription system 930.

In some embodiments, the notification suppression may be performed basedon identification at the presentation system 906 from where an inboundcall originates (e.g., using caller ID). For example, in response to theinbound call coming from a party other than the transcription system(e.g., from the remote device 910) the notification may be suppressedbut in response to the inbound call coming from the transcription system930, the notification may not be suppressed.

In some instances a double-forwarding situation may occur. For example,when call forwarding is enabled, calls that are directed toward thepresentation system 906 using the system number may be forwarded to thetranscription system 930 as discussed above. However, if thetranscription system 930 attempts to connect to the presentation system906 using the presentation system number, the associated call may beforwarded back to the presentation system 906. In some embodiments, thedouble-forwarding situation may be avoided using one or more techniquesas follows.

For example, the presentation system 906 may be linked to a firstpresentation system number and a second presentation system number.Calls directed toward the first presentation system number may beconfigured to be forwarded to the transcription system 930 as describedherein. However, calls directed toward the second presentation systemnumber may not be forwarded. As such, the transcription system 930 mayuse the second presentation system number to finish establishing thecommunication session. In these or other embodiments, the presentationsystem 906 may be configured to not ring when receiving a call to afirst presentation number and to ring when receiving a call to a secondpresentation number.

Additionally or alternatively, the first presentation system number maybe configured to connect to an inbound line at the transcription system930 (meaning that the first presentation system number is assigneddirectly to the center with no call forwarding). In these or otherembodiments, the second presentation system number may correspond to theoriginal presentation system number, a second phone line at thepresentation system 906, an ATA connected to the presentation system906, a digital phone of the presentation system 906 that connectsdirectly to an Internet port (i.e. a phone with a built-in ATA), or anapp such as a softphone or captioned softphone on a PC or smartphonethat are part of the presentation system 906.

Additionally or alternatively, the call forwarding may be configured toforward a given call over only one hop, so that the first inbound call(e.g., from the remote device 910) is forwarded to the transcriptionsystem 930, but the second inbound call (e.g., from the transcriptionsystem 930) is not forwarded. In these or other embodiments, instead ofallowing only one hop per call, the call forwarding service may beconfigured to allow only a single call within a particular period oftime to be forwarded so that all subsequent calls (including the inboundcall received from the transcription system 930) are directed toward thepresentation system 906.

In some embodiments, the call forwarding service configuration may havemany different parameters including one or more of: a forwarding limit(such as a maximum number of hops per call) may be configuredindividually for the subscriber account associated with the presentationsystem 906 and the corresponding linked phone number; the forwardinglimit may be configured for all subscribers in a selected pool, wherethe pool corresponds, for example, to IP Captioned Telephone Service(CTS) subscribers; and the forwarding limit may be configured for allcustomers of a particular telephone service provider.

Additionally or alternatively, the call forwarding service may beconfigured to activate call forwarding only for calls meeting (or,conversely, only for calls failing) one or more selected criteria suchas a rule applied to the calling number used to initiate the inboundcall. For example, call forwarding may be configured to activate ordeactivate forwarding with respect to calling numbers that are includedin a selected set, such as a one or more transcription system numbers.For instance, the call forwarding service may bypass call forwarding forcertain inbound calls in response to the ANI or Caller ID of the inboundcalls indicating that they are from the transcription system 930. Theone or more numbers that bypass call forwarding may be provided to thecall forwarding service by the transcription system 930 and/or thepresentation system 906. Conversely, a set of numbers may be specifiedfor which call forwarding is activated, so that calls from all othernumbers (including those from the transcription system 930) are notforwarded and instead go directly to the presentation system 906.

As another example, when initiating the second call to finishestablishing the communication session, the transcription system 930 maysend a message to the call forwarding service requesting that it notforward the second call. The request may include one or more identifiersfor the second call such as a session ID, IP address, or phone number sothat the call forwarding service knows which call to not forward. Thisrequest may be part of the second call or it may be via a separatecommunication with the call forwarding service. For example, a messageto the forwarding service (such as one or more SIP messages, APImessages, or a prefix or suffix of dialed digits) may bypass(deactivate) call forwarding for that call. For example, if thedeactivation (a.k.a. “call forward override”) string is *83 #, then thetranscription system 930 may dial *83 #followed by the presentationsystem number to bypass call forwarding for that particular call.

Additionally or alternatively, the call forwarding may be configured tobypass call forwarding in response to the inbound call being from thecall forwarding number. For example, suppose call forwarding is set upto send calls to 1-987-654-3210, which sends the call to thetranscription system 930. If the call forwarding service sees a callfrom the call forwarding number (e.g., 1-987-654-3210), it rings thepresentation system 906 instead of forwarding the call to thetranscription system 930. Additionally or alternatively, the callforwarding may be configured to bypass call forwarding in response tothe inbound call being from one or more selected numbers associated withthe transcription system 930.

In these or other embodiments, call forwarding may be configured toprevent any calling pattern that creates a potential repeating loop,such as the presentation system 906 and the transcription system 930repeatedly forwarding a call to each other.

Additionally or alternatively, the call forwarding service may beconfigured to ring the dialed number in response to the call forwardingnot being successful. In these or other embodiments, the transcriptionsystem 930 may reject a call when forwarded a second time, causing thecall forwarding service to ring the presentation system number. Forexample, an inbound call using the presentation system number may beforwarded to the transcription system 930, which may then attempt tofinish establishing the communication session by calling thepresentation system 906. The call forwarding service may attempt toforward the second call back to the transcription system 930. Thetranscription system 930 may detect that the second call is associatedwith the inbound call and may reject it to block the call forwarding(e.g., by presenting a busy signal, a reorder signal, or some signalother than ring and answer that blocks call forwarding). The callforwarding service may detect that the call forwarding attempt failedand may bypass call forwarding for the second call so that thepresentation system 906 receives the second call initiated by thetranscription system 930.

In these or other embodiments, before initiating the second call to thepresentation system 906, the transcription system 930 may brieflydisable call forwarding. For example, call forwarding control may beaccomplished using a phone call to the service provider, via an API, orvia a web interface, as described above. At some point after the secondcall is placed to the presentation system 906 (such as after the call isplaced, a phone of the presentation system 906 begins to ring, or thephone is answered), the transcription system 930 may re-enable callforwarding. In these or other embodiments, the call forwarding may beenabled or disabled using a “Remote Call Forwarding” feature offered bythe service provider that may be reached via an access number. TheRemote-Call Forwarding feature may receive the presentation systemnumber or a username, a password or PIN associated with a subscriber whocorresponds to the presentation system number, and a command withrespect to enablement of call forwarding (e.g. to enable call forwardingor disable call forwarding).

In these or other embodiments in response to an inbound call, thepresentation system 906 instead of the transcription system 930 maydisable, then re-enable call forwarding. In some embodiments, thepresentation system 906 may disable the call forwarding in response tothe call forwarding service providing an alert, such as a single quickring or a message, that a call was received and forwarded. In responseto such an alert, the presentation system 906 may temporarily disablecall forwarding (e.g., after the first inbound call has already beenforwarded) so that it may then receive the subsequent call from thetranscription system 930. The presentation system 906 may then re-enablecall forwarding using any suitable mechanism. For example, thepresentation system 906 may enable and/or disable call forwarding usingSIP messages, a smartphone app (or app running on some other device), anAPI, a web site interface, or a separate phone call placed after thealert is received (to disable) and after the first call has ended (toenable).

As another example, in some embodiments, the call forwarding may be setto be conditional on whether a call is answered. In these or otherembodiments, the presentation system 906 may be configured to ignoreincoming calls (and may optionally mute ringing) unless caller IDindicates that they are from the transcription system 930. Inbound callsthat are not from the transcription system 930 may therefore be ignoredand eventually forwarded to the transcription system 930. Variousmethods exist for ignoring incoming calls, including one or more of: notanswering the call, a phone of the presentation system 906 pretending tobe turned off or out of range or in airplane mode, and pretending to bebusy.

In these or other embodiments, inbound calls from the transcriptionsystem 930 may be recognized by the presentation system (e.g. based onthe calling phone number of the inbound call) and the presentationsystem 906 may accept the call. In some embodiments, the phone of thepresentation system 906 may continue to ring (even though it has alreadyautomatically answered the call) until a user answers the phone.Additionally or alternatively, the presentation system 906 may detectwhen an inbound call is not from the transcription system 930 and mayeither act (such as a flash-hook transfer) to transfer the call to thetranscription system 930 or establish a three-way call with thetranscription system 930 and the remote device 910. In some embodiments,while the three-way or forwarding is being processed, the phone of thepresentation system 906 may continue to ring until it is answered.

As another example, in some embodiments, the phone of the presentationsystem 906 may go off-hook or otherwise pretend to be busy and activatea “call forwarding on busy” mode and remain in that state until aninbound call is received. “Call forwarding on busy” may then forwardinbound calls to the transcription system 930. In these or otherembodiments, the presentation system 906 may detect inbound calls viacall waiting (e.g., the presentation of a “beep” when a second call iscoming in) and then either deactivate call waiting or end the busysession (e.g. go back on hook) so that it will receive the secondinbound call from the transcription system 930.

As another variation, in some embodiments, the presentation system 906may, at startup and at selected times such as whenever it is idle, placea static call (i.e. the call remains in place indefinitely) to thetranscription system 930 and leave the call up. When an inbound call isreceived (e.g., from the remote device 910), it may be forwarded (forexample because the line is busy) to the transcription system 930. Inthese or other embodiments, the transcription system 930 may bridge theforwarded call to the static call using any suitable technique such asthose described above. In some embodiments, the static call may continueafter the communication session with the remote device 910 ends.

In examples above where the transcription system 930 presents a callerID, ANI, or other phone number or device identifier associated with thetranscription system 930 (e.g., to disable call forwarding), thepresentation system 906 may still display the phone number of theoriginal calling party (e.g., the remote device number linked to theremote device 910). In some embodiments this may be accomplished bysending the presentation system 906 the original number as a secondmessage from the transcription system 930. For example, thetranscription system 930 phone number may be transmitted between thefirst and second ring and the original number may be transmitted betweenthe second and third ring. Additionally or alternatively, the originalnumber may be sent as part of a data message (e.g. via an API to an apprunning on the presentation received) or on a separate audio or datachannel.

In some instances, the telephone devices of the presentation system 906may not be initially configured to be able to present transcriptions.For example, the presentation system 906 may be located at a particularlocation that is a particular type of location (e.g., a business, agovernment office, a rest home, etc.) where the telephones are mandatedas being a particular type and/or tied to a specific network and may notsupport transcription services. In these or other embodiments, thepresentation system 906 may include a relay center that may be set upfor the particular location and that may be configured to performoperations that help allow for transcription services to be performed.

For example, in some embodiments, the relay center may be configured tocapture the audio of the communication session (e.g., the first audioand/or the second audio described above) and communicate the audio tothe transcription system 930. In some embodiments, the audio may becaptured by a PBX of the relay center. In these or other embodiments,the audio may be captured by the relay center using a SIP LawfulIntercept (LI). In the case of business phones, the LI capability mayreside on a PBX or other local switch of the relay center. Forresidential phones, LI may reside in the service provider's network.

Additionally or alternatively, the relay center may set up and/or host athree-way call between the participating device of the presentationsystem 906 (e.g., the telephone that is participating in thecommunication session), the remote device 910, and the transcriptionsystem 930 so that the transcription system 930 may listen to the calland obtain the corresponding audio. Alternatively, the three-way callmay be hosted by the service provider network.

In some embodiments, the bridging with the transcription system 930 maybe requested by the participating device of the presentation system 906.For example, the participating device may send a SIP message to therelay center or to a telecom switch requesting the bridge be set up.Alternatively or additionally, the participating device may send amessage to the transcription system 930 with a conference bridge URI andthe transcription system 930 may send a SIP INVITE to the participatingdevice and/or the remote device 910 and may identify the URI of thedialog. In some embodiments, communication of the data (e.g., forpassing the URI, SIP requests, and other connection requests describedabove) may use any suitable data network or one of the methods describedbelow for providing transcriptions to the presentation system 906.Alternatively or additionally, once on a three-way call, the relaycenter, and/or the presentation system 906 (e.g., the participatingdevice of the presentation system 906) may be configured to inject datasignals into the conference bridge audio channel (e.g., such asdescribed in further detail below).

As another example, the presentation system 906 may include a particulardevice (e.g., a handset, an external display device, etc.) that may beprovided for telephones of the particular location in which theparticular device may be configured to establish a data connection withthe transcription system 930 to communicate audio to the transcriptionsystem 930 and/or receive transcriptions from the transcription system930.

In these or other embodiments, the particular device of the presentationsystem 906 may include a tap system that may be inserted inline with aphone handset cord or the telephone line connected to the telephone. Thetap system may be configured to intercept the audio of the communicationsession and communicate the audio to the transcription system 930. Inthese or other embodiments, the tap system may also be configured toperform other functions such as (a) translate DTMF signals to call thetranscription system 930 instead of the number the user dials, (b) sendthe number dialed by the user to the transcription system 930 so that itmay bridge or relay the call to the device associated with the dialednumber (e.g., the remote device 910), (c) receive transcriptions fromthe transcription system 930, (d) block DTMF or other data signals sothat they are not heard by a user of the presentation system 906, (e)separate an audio stream from a data stream (e.g., separate audio fromtranscriptions in instances in which both are communicated over a samecommunication connection, such as discussed below), (f) send thetranscription to a display of the transcription presentation system 914,and (g) control the display such as showing information, graphics, andbuttons and receiving information from the display. In some embodiments,the tap system may be analogous to the tap system described above withrespect to FIGS. 6 and 7 .

Additionally or alternatively, the tap system may be a third-partydevice such as an ECHO CONNECT® that is configured to intercept theaudio and relay the audio to the transcription system 930. In these orother embodiments, the transcription system 930 may communicate thetranscription back to the ECHO CONNECT, which may then communicate thetranscription to another device of the transcription presentation system914 (e.g., a television, a computer monitor, a smartphone, a tabletcomputer, an ECHO SHOW® device, an ALEXA SPOT® device, etc.). In someembodiments, the integration with such an example AMAZON® system mayinclude providing one or more ALEXA® skills to configure the system toperform such functionality. Similar operations may be performed withrespect to any other suitable smart device system, such as a GOOGLEHOME® system.

In these or other embodiments, the particular device may include a boxthat may be plugged into a headset or other port of a phone of thepresentation system 906. The box may be configured to relay audio to thetranscription system 930 and/or receive the transcription from thetranscription system 930.

In these or other embodiments, the particular device may include astand-alone screen for presenting the transcription. The screen mayreceive the transcription from the transcription system 930 via aseparate network connection (e.g. WiFi), from the handset, the tapsystem etc., in some embodiments. In these or other embodiments, theparticular device may be an existing screen such as a television or acomputer monitor, which may be set up to receive the transcriptionaccording to any suitable technique.

Additionally or alternatively, software may be provided to a telephoneof the presentation system 906 in which the software configures thephone to perform transcription services. In some embodiments, thesoftware may be configured to cause the phone to communicate with thetranscription system 930 to perform the transcription services.Additionally or alternatively, the software may configure the phone totranscribe the audio into the transcription (e.g., using ASR). In theseor other embodiments, the software may enable the presentation of thetranscription on a display of the phone. For example, the software maybe an app that runs on a CISCO® phone that configures the phone to beable to have such functionality. Additionally or alternatively, suchsoftware may be configured for and ported to existing IP videophoneslike the Mitel 6873, Grandstream GXV3275, or Cisco DX650. In these orother embodiments, a softphone software application (e.g., X-Lite) thatmay be run on any suitable device (e.g., PC, tablet, smartphone, etc.)may provide one or more elements of the above-noted functionality. Inthese or other embodiments, web-based communication services such asSKYPE®, GOGGLE® Voice, FACETIME®, etc. may be used to conduct thecommunication session. Additionally or alternatively, the web-basedcommunication services may be configured to intercept the audio of thecommunication session and to communicate the audio to the transcriptionsystem 930. In these or other embodiments, the transcription system 930may be configured to communicate the transcription to the correspondingweb-based communication service, which may be configured to communicatethe transcription to the presentation system 906.

As indicated above, in some embodiments, the call forwarding may beactivated and/or hosted using a call forwarding server. FIG. 9Billustrates an example environment 950 for implementing call forwardingusing a call forwarding server 952. The environment 950 may be arrangedin accordance with at least one embodiment described in the presentdisclosure. The environment 950 may include the presentation system 906,the transcription system 930, the remote device 910, the call forwardingserver 952, a service provider 954, and a Secondary Telephone Server(STS) 956, which may be communicatively coupled in any suitable manner.For example, the elements of FIG. 9B may be communicatively coupled viathe first network 902 and/or the second network 904 of FIG. 9A (notexpressly illustrated in FIG. 9B).

The call forwarding server 952 may include any suitable hardware and/orsoftware configured to perform the operations described herein withrespect to the call forwarding server 952. For example, the callforwarding server 952 may include code and routines configured to enablea computing device to perform one or more of the described operations.Additionally or alternatively, the call forwarding server 952 mayinclude one or more processors and one or more computer-readable media.

The STS 956 may include any suitable hardware and/or software configuredto perform the operations described herein with respect to the STS 956.For example, the STS 956 may include code and routines configured toenable a computing device to perform one or more of the describedoperations. Additionally or alternatively, the STS 956 may include oneor more processors and one or more computer-readable media.

The service provider 954 may include any suitable system or device,including hardware and software, relay devices, base stations,communication endpoints, etc., configured to provide telecommunicationservices. The service provider 954 may utilize any suitable network toprovide the telecommunication services.

An example of the operation of the environment 950 is now provided. Insome embodiments, the presentation system 906 (e.g., a device of thepresentation system 906 or a software application on the device) oranother system such as the transcription system 930 may communicate amessage to the call forwarding server 952 (e.g., via an audio connectionor a data connection). The message may include information that may beused to set up or end call forwarding. For example, the information mayinclude the presentation system number, or another suitable identifierof the presentation system 906 or an associated device of thepresentation system 906, a carrier code associated with the serviceprovider 954, dialing strings used for call forwarding, API messages,website interface signals, DTMF tones, etc.

In response to the message, the call forwarding server 952 maycommunicate with the service provider 954 to instruct the serviceprovider 954 to activate call forwarding according to the informationincluded in the message. The communication may be performed using anysuitable analog or digital protocol. For example, in some embodimentsthe communication may be performed over an analog audio line and thecall forwarding server 952 may send a series of DTMF signals to activatethe call forwarding. Additionally or alternatively, the communicationmay be performed over a data network and may be communicated via an APIor web site associated with the service provider 954.

In some embodiments, the call forwarding server 952 may interact withthe website as if it were the customer of the service provider 954 (whomay be also be associated with the presentation system 906). Forexample, the call forwarding server 952 may be configured to mimic thecustomer's actions by, for example, screen-scraping a web page of theweb site to obtain information from the service provider 954 andinteracting with the web page (e.g., clicking buttons or otherwiseposting information) to provide information to the service provider 954that is related to the call forwarding. In some embodiments, thecustomer's web site login credentials (e.g. login name, PIN, password)may be stored on the call forwarding server 952 or elsewhere that may beaccessible by the call forwarding server 952 to enable the callforwarding server to provide information on the web site on behalf ofthe customer.

As another example, the call forwarding server 952 may spoof its phonenumber as being that of the presentation system 906 and may dial a codesuch as *72 plus the forwarding phone number associated with thetranscription system 930 to initiate the call forwarding. In these orother embodiments, the call forwarding server 952 may also set upmulti-ring, or sequential ringing as described below.

In some embodiments, a software application of the presentation system906 may operate as the call forwarding server 952 and may communicatedirectly with the service provider 954 to perform the describedoperations of the call forwarding server 952 using any suitable digitalor analog protocol or process such as those described above.

After the call forwarding has been enabled, for an inbound call from theremote device 910, the service provider 954 may forward the inbound callto the STS 956 such that an audio connection between the STS 956 and theremote device 910 may be established. In these or other embodiments, theSTS 956 may also establish audio connections with the transcriptionsystem 930 and with the presentation system 906 in response to the callbeing forwarded thereto. For example, in some embodiments, the STS 956may relay the first audio that originates at the remote device 910 tothe transcription system 930. In these or other embodiments, the STS 956may receive the transcription from the transcription system 930 andcommunicate the transcription to the presentation system 906 via a dataconnection such as described below. Additionally or alternatively, theSTS 956 may relay the audio that originates at the remote device 910 tothe presentation system 906. In some embodiments, the audio may berelayed to a device of the presentation system 906 or a softwareapplication (also referred to as an “app”) running on the device. TheSTS 956 may thus be configured to be part of and/or establish the firstaudio connection 940, the second audio connection 942, and/or the thirdaudio connection 944 described with respect to FIG. 9A.

In some instances, the relaying of the audio to the app may be becausethe operating system of the device (e.g., the operating system of asmartphone) may not easily allow apps to access the device's telephonefunction. Therefore, it may be useful for the app to be able tocommunicate with the STS 956 over a separate communication channel in amanner that may bypass the built-in phone functions of the device. Theapp may operate as a softphone, for example, sending audio to and fromthe STS 956 over a data channel, ringing, allowing the placement ofcalls, and performing other telephone functions.

As an alternative to the STS 956 forwarding calls, it may set up a 3-wayconference call. Additionally or alternatively, the STS 956 may not beinvolved in the communication of the first audio to the transcriptionsystem 930 and/or may not be involved in the communication of thetranscription to the presentation system 906.

In some embodiments, the call forwarding server 952 and the STS 956 maybe separate or they may be combined with each other. Additionally oralternatively, the call forwarding server 952 and/or the STS 956 may bepart of the transcription system 930 and may be integrated with otherelements of the transcription system 930 such as an automatic calldistributor (ACD), ASR systems, and other telephony systems.

In some embodiments, the communication of audio using the environment950 may be conducted via an app that may be stored on a device that ispart of the presentation system 906 (e.g., device may be a smartphoneand the app may be an app of the smartphone). For example, the remotedevice 910 may be used to initiate an inbound call to the presentationsystem 906. The inbound call may be sent to the STS 956 (e.g., by callforwarding or simply because that is where all calls to the presentationsystem 906 are configured by the service provider 954 to go). The STS956 may answer the call and may connect to the app on the presentationsystem 906. The app may present a notification that a call has arrived.In some embodiments, the app may play a ringing signal until the call isaccepted. On acceptance, STS 956 may bridge audio between the app andthe remote device 910 and may also communicate the first audio and/orthe second audio to the transcription system 930. The transcriptionsystem 930 may generate the corresponding transcription and maycommunicate the transcription to the STS 956 in some embodiments. Inthese or other embodiments, the STS 956 may communicate thetranscription to the presentation system 906.

In some embodiments, the presentation system 906 may include multipledevices that may be used to conduct a call. For example, thepresentation system 906 may include a smartphone and a landline phone.As another example, the presentation system 906 may include an ATA thatmay be connected to a data network and that may provide a phone line tothe landline phone. In some embodiments, the ATA may send audio betweenthe remote device 910 and the presentation system 906 which may allowparties to communicate. Additionally or alternatively the ATA may sendaudio from the remote device 910 to the transcription system 930, whichmay allow the transcription system 930 to generate transcriptions. Sincethe audio from each caller may be separate in the ATA (and not mixedtogether as it might be in a telephone hybrid or in an analogtelephone), the ATA may obtain audio from the remote device 910 alonewithout use of echo cancelers or telephone hybrids. The ATA may alsosend audio from the presentation system 906 to the transcription system930 for generation of transcriptions. In these or other embodiments, theSTS 956 may be configured to connect to one more of the devices. In someembodiments, the STS 956 may connect to the devices using one or more ofthe following methods.

For example, in some embodiments, the STS 956 may multi-ring two or moreof the devices and may connect to whichever device answers the callfirst. Additionally or alternatively, the STS 956 may ring two or moreof the devices and two or more of the devices may be answered by theuser. In these or other embodiments, the STS 956 may set up a conferencecall with the transcription system 930, the remote device 910, and eachof the answered devices of the presentation system 906. In these orother embodiments, a first device of the presentation system 906 may beanswered and used for the communication of audio and/or video as part ofthe communication session and the STS 956 may also ring a second deviceof the presentation system 906. The second device may be configured toautomatically answer the call from the STS 956 and the STS 956 may beconfigured to communicate the transcription to the second device, whichmay present the transcription. Additionally or alternatively, thetranscription may be communicated to and presented by the first devicein addition to the second device. Additionally or alternatively, thepresentation system may place a call to the remote device and the STS956 may place a call to the second device.

In these or other embodiments, the STS 956 may be configured to firstattempt to connect with the first device (e.g., by ringing the firstdevice). If the first device is busy or if there is no answer after aselected number of rings or period of time, the STS 956 may then attemptto connect with the second device in a sequential manner. In these orother embodiments, the STS 956 may continue to ring the first devicewhile ringing the second device.

In these or other embodiments, the STS 956 may ring the first device(e.g. a POTS phone) and may be configured to simultaneously activate anapp of the second device (e.g., an app of a smartphone). In these orother embodiments, the first device may be used for the voice call andthe second device may present the transcription of the call. In someembodiments, the user may answer the first device and then may open theapp if a transcription is desired. In this latter instance, because theuser made the effort to open the app (in addition to answering thephone), the extra effort may be used as a feature in determining whetherthe user is certified and/or has a legitimate need for receivingtranscriptions. This feature may be used to determine that a user shouldreceive transcriptions, even though the user may not be otherwise fullycertified as an IP CTS user. In these examples such as described above,the STS 956 may communicate with the smartphone via the built-in phonefunction or with an app on the phone.

The above referenced multi-ring operations are not limited to instancesin which the STS 956 is involved and may be applicable in any othersuitable configuration or situation.

As illustrated, both the call forwarding server 952 and the STS 956 maycommunicate with the second device via any suitable connection (e.g. aphone call to a smartphone) or via a data (including voice over data)connection to an app. This may be useful in instances in which an app isunable to directly place a voice phone call via the telephone functionsof the second device, in instances in which it is convenient for the appto communicate via a data path to the call forwarding server 952, orwhen the interface to the service provider 954 uses an API rather than aDTMF-based call forwarding protocol.

Returning to FIG. 9A, the above description discusses techniques forrouting audio (e.g., the first audio that originates at the remotedevice 910 and/or the second audio that originates at the presentationsystem 906) to or through the transcription system 930 such that thetranscription system 930 may obtain the audio for the generation of acorresponding transcription. As such, after obtaining the first audioand/or the second audio, the transcription system 930 may be configuredto obtain (e.g., generate) a transcription of the first audio and/or thesecond audio using any suitable techniques such as those described abovewith respect to FIG. 1 . In these or other embodiments, thetranscription system 930 may be configured to communicate thetranscription to the presentation system 906 via the second network 904.

For example, in some embodiments, the transcription system 930 and thetranscription presentation system 914 may be communicatively coupled viaa data connection 946 that is established over the second network 904.In these or other embodiments, the transcription system 930 maycommunicate the transcription to the transcription presentation system914 using the data connection 946.

The data connection 946 may be any suitable connection that may beestablished over the second network 904 for the communication of thetranscription. For example, the data connection 946 may include anysuitable wide area network connection such as an IP based connection, anaudio connection, a cellular network connection, etc.

In some embodiments, the second network 904 may include a short-rangecommunication network. For example, the second network 904 may be acombination of the second network 204 and the third network 206 of FIG.2 . In these or other embodiments, the data connection 946 may includeany suitable short-range communication network connection. Additionallyor alternatively, in some embodiments, the short-range communicationnetwork may be part of or implemented by the transcription presentationsystem 914 of the presentation system 906.

Below are some example embodiments of the transcription presentationsystem 914 and corresponding operations that may be performed by thetranscription presentation system 914 to obtain and/or present thetranscription.

In some embodiments, the transcription presentation system 914 mayinclude a display system that may be configured to receive cellularcommunications (e.g., via the second network 904). In these or otherembodiments, the data connection 946 may be via a corresponding cellularnetwork and the display system may receive the transcription via thedata connection 946 and may present the transcription on an associateddisplay. In some embodiments, the display system may include a SIM slotconfigured to receive a SIM card such that the display system mayreceive the cellular communications.

Additionally or alternatively, the transcription presentation system 914may include a hotspot-type device and a display. The hotspot-type devicemay be configured to receive cellular communications and the dataconnection 946 may be at least partially established via thecorresponding cellular network. The hotspot-type device may beconfigured to receive the transcription via the data connection 946.Further the hotspot-type device may be communicatively coupled to thedisplay and may communicate the transcription to the display forpresentation by the display. In some embodiments, the hotspot-typedevice may communicate the transcription over a short-range wirelessnetwork such as the second network 204 of FIG. 2 . In these or otherembodiments, the second device 214 of FIG. 2 may be an example of thehotspot-type device. Additionally or alternatively, the hotspot-typedevice may communicate the transcription to the display via a wiredconnection.

Additionally or alternatively, the transcription presentation system 914may include a router-type device and a display. The router-type devicemay be configured to receive communications over the Internet and thedata connection 946 may be at least partially established via theInternet. The router-type device may be configured to receive thetranscription via the data connection 946. Further the router-typedevice may be communicatively coupled to the display and may communicatethe transcription to the display for presentation by the display. Insome embodiments, the router-type device may communicate thetranscription over a short-range wireless network such as the secondnetwork 204 of FIG. 2 . Additionally or alternatively, the router-typedevice may communicate the transcription to the display via a wiredconnection.

Additionally or alternatively, in some embodiments the data connection946 and the first audio connection 940 may be part of a same particularcommunication channel. As such, in these or other embodiments, thetranscription may be communicated in conjunction with the first audioover the particular communication channel. In some embodiments, thecommunication of the transcription and the first audio together over thesame particular communication channel may be performed as describedbelow with respect to FIGS. 11-13 . In these or other embodiments, theparticular communication channel may be an analog communication channel.For example, in some embodiments, the particular communication channelmay be part of an analog voice network. Additionally or alternatively,the first particular communication channel may be configured topropagate digital communications.

In these or other embodiments, the transcription presentation system 914may include the display system, which may be configured to distinguishbetween and identify the transcription and the audio in instances inwhich the first audio connection 940 and the data connection 946 arepart of the same particular communication channel, such as described indetail below. In these or other embodiments, the display system may beconfigured to present the transcription as distinguished from the audio.In some embodiments, the display system may be part of the same devicethat receives the audio. Additionally or alternatively, the displaysystem may be separate from the device that receives the audio. In someembodiments, the display system may be integrated with a particulardevice of the presentation system 906 that also includes or is part ofthe audio system 912. In these or other embodiments, the display systemmay be separate from the particular device and may be communicativelycoupled to the particular device using an API, and/or any suitablewireless and/or wired connection.

Additionally or alternatively, the transcription presentation system 914may include a tap system such as the tap system described with respectto FIGS. 6 and 7 . In these or other embodiments, the tap system may beconfigured to distinguish between and identify the transcription and theaudio in instances in which the first audio connection 940 and the dataconnection 946 are part of the same particular communication channel. Inthese or other embodiments, the tap system may be configured tocommunicate the audio to the audio system 912 of the presentation systemand may be configured to communicate the transcription to one or moredisplay systems of the transcription presentation system. In someembodiments, the tap system may be configured to filter out thetranscription from the audio sent to the audio system 912 and to filterout the audio from the transcription sent to the one or more displaysystems.

In some embodiments, the tap system may be communicatively coupled to afirst display system of the transcription presentation system 914 and aparticular device of the presentation system that may include a seconddisplay system and a particular audio system, and that may be part ofthe audio system 912 and the transcription presentation system 914. Inthese or other embodiments, the tap system may be configured tocommunicate the transcription to the first display system andcommunicate the audio with the transcription embedded therein to theparticular device. In some embodiments, the tap system may re-embed thetranscription with the audio after separating the two out. Additionallyor alternatively, the tap system may relay the audio with thetranscription embedded therein prior to separating the two. In these orother embodiments, the particular device may be configured todistinguish between and identify the audio and the transcription and maybe configured to present the audio via its particular audio system andmay present the transcription via the second display system.Additionally or alternatively, the tap system may be configured to sendthe identified audio to the particular audio system of the particulardevice and may be configured to send the identified transcription to thesecond display system of the particular device.

In some embodiments, the tap system may be inserted inline with respectto a line cord of a phone line that is connected to the presentationsystem 906. In these or other embodiments, the tap system may beinserted inline with respect to a handset cord of a handset of atelephone of the presentation system 906. In these or other embodiments,the tap system may be connected to a headset output of the telephone.Additionally or alternatively, the tap system may be integrated with thehandset, headset, and/or the base of the telephone. The tap system maybe configured to communicate with any other applicable device or systemusing any suitable wired or wireless connection and associatedprotocols.

In these or other embodiments, the tap system may be configured toreceive the transcription over the data connection 946 in instances inwhich the data connection 946 is separate from the first audioconnection 940 (e.g., in instances in which the first audio connection940 and the data connection 946 are not part of a same communicationchannel). In these or other embodiments, the tap system may beconfigured to communicate the transcription to any suitable displaysystem using any suitable wired or wireless connection and associatedprotocols.

In some embodiments in instances in which the first audio connection 940and the data connection 946 are part of the same particularcommunication channel, the data connection 946 and the first audioconnection 940 may be digital connections, but the audio system 912 maybe or include an analog system (e.g., the audio system 912 may be partof an analog telephone). In these or other embodiments, thetranscription may be communicated to a first display system (e.g., via arouter configured to distinguish between and identify the transcriptionand the audio) that is separate from the analog telephone. In these orother embodiments, the audio and/or transcription may be communicated(e.g., via the router) to an analog telephone adapter (ATA) configuredto convert digital signals to analog signals. The ATA may becommunicatively coupled to and/or part of the analog telephone and maycommunicate the now analog audio and transcription to the analogtelephone. In some embodiments, the ATA may be configured to distinguishbetween and identify the audio and the transcription and may send theidentified audio and the identified transcription to the analogtelephone for presentation. In these or other embodiments, the ATA maysend the audio with the transcription embedded therein and the analogtelephone may be configured to distinguish between and identify theaudio and the transcription to identify the audio from thetranscription.

The environments 900 and 950 may accordingly be configured to routeaudio through or to the transcription system 930 and the presentationsystem 906 may be configured to receive and present correspondingtranscriptions generated by the transcription system 930 as describedabove. The above-related description may allow for more ability toprovide transcription services in environments in which transcriptionservices may not be readily available or capable of being performed.Modifications, additions, or omissions may be made to the environments900 and 950 and/or the components operating in the environments 900 and950 without departing from the scope of the present disclosure. Forexample, in some embodiments, the environment 900 or the environment 950may be integrated into other environments that provide additionalbenefits for a user. As another example, the particular arrangement anddescription of the components are merely examples used to help explainthe concepts described herein and are not meant to be limiting.

FIG. 10 illustrates an example environment 1000 for communicating atranscription and corresponding audio over a same communication channel.The environment 1000 may be arranged in accordance with at least oneembodiment described in the present disclosure. The environment 1000 mayinclude a network 1002, a transcription system 1030, a remote device1010, and a presentation system 1006.

In some embodiments, the network 1002 may be analogous to the network102 of FIG. 1 . In the illustrated example of FIG. 10 , the network 1002may be configured to communicatively couple the presentation system1006, the remote device 1010, and the transcription system 1030. In someembodiments, the network 1002 may include an analog network, such as ananalog voice network. Additionally or alternatively, the network 1002may include a digital network.

Additionally or alternatively, the network 1002 may include acommunication channel 1068 that may be used to communicate information(e.g., audio and/or a transcription of a communication session). Forexample, the communication channel 1068 may be a phone line of an analogvoice network. In the example of FIG. 10 , the arrows and linesillustrated as representing the communication channel 1068 are merely tohelp with visualizing that the communication channel 1068 is between thepresentation system 1006 and the transcription system 1030. The arrowsand lines are not meant to represent the actual path of thecommunication channel 1068.

The transcription system 1030 may be similar or analogous to thetranscription system 130 of FIG. 1 and may be configured to generate atranscription 1060 based on audio 1062 of a communication session thatmay be conducted between the presentation system 1006 and the remotedevice 1010, which may be analogous to the remote device 110 of FIG. 1 .The transcription system 1030 may be configured to generate thetranscription 1060 using any suitable technique such as those describedin the present disclosure. Reference to the transcription 1060 may alsorefer to any suitable signal that may be used to communicate thetranscription 1060 and associated data.

As indicated above, the audio 1062 may include audio that may beassociated with the communication session between the presentationsystem 1006 and the remote device 1010. For example, the audio 1062 mayinclude first audio that may originate at the remote device 1010 and bereceived and presented at the presentation system 1006 (e.g., via theaudio system 1012) during the communication session. Additionally oralternatively, the audio 1062 may include second audio that mayoriginate at the presentation system 1006 and be received and presentedat the remote device 101 during the communication session. In someembodiments, the audio 1062 may be routed to or through thetranscription system 1030 using any suitable technique such as thosedescribed above with respect to FIGS. 9A and 9B. In the presentdisclosure, reference to the audio 1062 may also refer to any suitablesignal that may be used to communicate the audio 1062 and associateddata.

In some embodiments, the transcription system 1030 may include a firstsignal processing system 1064. The first signal processing system 1064may include any suitable hardware and/or software configured to processthe transcription 1060 and/or the audio 1062. In the present disclosureany operation that may be performed by the first signal processingsystem 1064 with respect to the transcription 1060 or the audio 1062 maybe considered as “processing” the transcription 1060 or the audio 1062to obtain a “processed” transcription 1060 or “processed” audio 1062.Reference to the processed transcription 1060 or the processed audio1062 in the present disclosure may also refer to the signals that may beconfigured to carry the information associated with the processedtranscription 1060 or the processed audio 1062. Further, in someinstances, reference to the audio 1062 or the transcription 1060 mayinclude instances in which the audio 1062 or the transcription 1060 maybe considered processed audio 1062 and the processed transcription 1060.For example, as discussed in detail below, the audio 1062 and thetranscription 1060 may be multiplexed into combined data. As such,reference of the audio 1062 and the transcription 1060 with respect tothe combined data may also be referring the processed audio 1062 and theprocessed transcription 1060 even if not explicitly stated as such.

The operations that may be performed by the first signal processingsystem 1164 may be referred to as “first processing operations” and mayinclude one or more operations that may include analysis operations,encoding operations, modulating operations, filtering operations, datacompression operations, frequency and/or time multiplexing operations,signal relaying operations, signal routing operations, bandwidthcompression operations, frequency shifting, phase shifting, signalstorage, delay, multipliers, amplification, data and/or signalcompression, speech enhancement, quantization, smoothing, interpolation,table lookups, linear or non-linear transformation, rectification,normalization, etc., or any suitable combination thereof. For example,the first signal processing system 1064 may include one or moreswitches, encoders, analog filters, digital filters, multiplexers,digital signal processing systems, neural networks, signal routers,modems, etc., or any suitable combination thereof configured to performone or more of the first processing operations. In some embodiments, thefirst signal processing system 1064 may be configured to perform one ormore of the first processing operations as described below with respectto a first signal processing system 1164 of FIG. 11 , which may be anexample of the first signal processing system 1064.

In general, the first signal processing system 1064 may be configured tomultiplex the audio 1062 and the transcription 1060 to generate thecombined data. The combined data may thus include the audio 1062 and thetranscription 1060 (e.g., as processed audio 1062 and the processedtranscription 1060) and may be communicated to the presentation system1006 over a same communication channel of the network 1002. For example,the combined data may be communicated to the presentation system 1006over the communication channel 1068 such that the presentation system1006 may receive both the audio 1062 and the transcription 1060 over thecommunication channel 1068. Although referred to as “data,” reference tothe combined data may also refer to any suitable signal that may be usedto carry the information that may be included in the combined data.

The presentation system 1006 may be similar or analogous to thepresentation system 906 of FIGS. 9A and 9B. For example, thepresentation system 1006 may include an audio system 1012 that may beanalogous to the audio system 912 of FIG. 9A. Additionally, thepresentation system 1006 may include a transcription presentation system1014 that may be analogous to the transcription presentation system 914of FIG. 9A. The presentation system 1006 may also include a userinterface (not expressly illustrated in FIG. 10 ) analogous to the userinterface 916 of FIG. 9A.

In some embodiments, the presentation system 1006 may include a secondsignal processing system 1066. The second signal processing system 1066may include any suitable hardware and/or software configured to processthe combined data generated by the first signal processing system 1064to reproduce the audio 1062 and the transcription 1060 from the combineddata.

In the present disclosure any operation that may be performed by thesecond signal processing system 1066 to reproduce the transcription 1060or the audio 1062 from the combined data may be considered as“processing” the combined data, the transcription 1060, or the audio1062 to obtain “reproduced” data. In some embodiments, the operationsperformed by the second signal processing system 1066 may be referred toas “second processing operations” and may include one or more operationsthat may include decoding operations, demodulating operations, filteringoperations, data de-compression operations, frequency and/or timede-multiplexing operations, signal relaying operations, signal routingoperations, bandwidth extension operations, frequency shifting, phaseshifting, signal storage, delay, multiplication, amplification, dataand/or signal de-compression, speech enhancement, noise reduction,quantization, smoothing, interpolation, table lookups, linear ornon-linear transformation, rectification, normalization, etc., or anysuitable combination thereof. For example, the second signal processingsystem 1066 may include one or more switches, decoders, analog filters,digital filters, multiplexers, digital signal processing systems, neuralnetworks, signal routers, modems, etc., or any suitable combinationthereof configured to perform one or more of the second processingoperations.

The second processing operations of the second signal processing system1066 may in general be configured to distinguish between and identifythe transcription 1060 and the audio 1062 included in the combined datato reproduce the transcription 1060 and the audio 1062 from the combineddata. In these or other embodiments, the second signal processing system1066 may be configured to communicate the audio 1062, as reproduced fromthe combined data, to the audio system 1012 for presentation by theaudio system 1012. Additionally or alternatively, the second signalprocessing system 1066 may be configured to communicate thetranscription 1060, as reproduced from the combined data, to thetranscription presentation system 1014 for presentation by thetranscription presentation system 1014. In some embodiments, the secondsignal processing system 1066 may be configured to perform one or moreof the second processing operations as described below with respect to asecond signal processing system 1166 of FIG. 11 , which may be anexample of the second signal processing system 1066.

In some embodiments, the presentation system 1006 may include multipledevices that may be connected to the communication channel 1068. Forexample, the communication channel 1068 may be a particular telephoneline and the presentation system 1006 may include multiple telephonesconnected to the particular telephone line. In these or otherembodiments, the presentation system 1006 may be configured to reduce orprevent the presentation, at the other telephones, of sounds that arerelated to the communication of the transcription 1060 over theparticular telephone line (e.g., as the processed transcription 1060included in the combined data).

For example, as indicated below with respect to FIG. 11 , in someembodiments, the processed transcription 1060 may be communicated usingDTMF signaling. In these or other embodiments, one or more of thetelephones of the presentation system 1006 may be configured to detectand block DTMF signals from being communicated to their correspondingaudio systems (e.g., earpieces) such that the DTMF signals thatcorrespond to the processed transcription 1060 may not be presented.

In some instances, there may be some delay in the DTMF detection. Assuch, in some embodiments, a short (e.g., −2-40 ms) delay may be appliedto presentation of the audio in the audio signal heard by the user togive the DTMF detector time to respond. DTMF signaling may notexperience a lot of interference by other extensions, such thatrelatively simple methods of error detection and correction such asretransmitting lost data may be employed.

In these or other embodiments, filters may be implemented in the othertelephones to remove the data signal of the combined data that maycorrespond to the transcription 1060. For example, if the transcription1060 is sent in the 3 kHz-4 kHz band (e.g., according to one or moretechniques described below with respect to FIG. 11 ), the filter mayremove that band from the signal (e.g., the combined data) that may bereceived by the filter such that sounds that correspond to communicationof the transcription 1060 may not be presented at the correspondingphone. In another example, filters may mute DTMF tones that may be usedto communicate the transcription 1060.

In these or other embodiments, inline filters may also be connected tothe other telephones that may be connected to the communication channel1068 in a manner similar to how inline DSL filters are used Inlinefilters for the other telephones may receive and respond to signals fromthe second signal processing system 1066 (e.g., as included in aparticular device (“device1”) that includes the second signal processingsystem 1066 (e.g., a captioning telephone)) on how best to removeinterference at a given time. For example, the second signal processingsystem 1066 may send a data signal in high frequencies (e.g. above 4 kHzand therefore inaudible if filtered out) and use the inline filters toremove the distortion. If the inline filter uses powered electronics, asmall amount of current may be extracted from the communication channel1068 (e.g., as a phone line), a battery (or a supercapacitor) charged byphone line power, or from an axillary power supply.

In these or other embodiments, the device1 may be configured to listento the data signal of the transcription 1060 (“transcription datasignal”) and then transmit a signal on the wires of the telephone lineof the communication channel 1068 to cancel the data signal to the otherphones. Additionally or alternatively, the device1 may be configured tosend a clean copy of the audio 1062 of the combined data, shifted tohigh frequencies (e.g. above 4 kHz), then frequency-shifted back to amore normal (e.g. 0-4 kHz) audio band by inline filters and used tocancel the transcription data signal output by the filters to otherdevices connected to the line. The filters may delay the transcriptiondata signal (contained in the baseband 0-4 kHz) or the frequency-shiftedsignal so that both signals are time-aligned.

Additionally or alternatively, the device1 may be configured tofrequency-shift the audio signal of the combined data to a band above 4kHz and may send the frequency shifted audio signal over the phone line.A filter inline with one or more other phones connected to the line maybe configured to attenuate the 0-4 kHz band so that the transcriptiondata signal is removed or substantially removed (e.g., removed to bebelow a particular threshold power). In these or other embodiments, theinline filter may be configured to frequency-shift the audio signal backto the 0-4 kHz band and send it to the corresponding phone that iscoupled to the inline filter. The function described for the filter mayalternatively be contained in the other phones instead of being insertedas a separate device in the phone lines.

In these or other embodiments, the device 1 may be configured to detectthat another phone connected to the phone line has been picked up oranswered, for example by detecting a drop in line voltage (e.g., of aPOTS line). In these or other embodiments, the device1 may switch to adifferent mode of sharing the audio 1062 and transcription 1060 of thecombined data based on detecting another connected phone. For example,it may share bandwidth with the transcription signal (which anothercaller in the house might hear) when there are no other phones, thenswitch to sending the transcription 1060 quietly, such as duringsilence, when another phone is detected. The reverse process may beperformed when another phone is hung up or placed on-hook.

The environment 1000 may accordingly be configured to communicate boththe transcription 1060 and the audio 1062 over the communication channel1068. Such arrangement may allow for the conducting of transcriptionservices for locations that may not have other data communication access(e.g., Internet, cellular network) for the reception of thetranscription 1060.

Modifications, additions, or omissions may be made to the environment1000 and/or the components operating in the environment 1000 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the environment 1000 may be integrated into otherenvironments that provide additional benefits for a user. As anotherexample, the particular arrangement and description of the componentsare merely examples used to help explain the concepts described hereinand are not meant to be limiting.

FIG. 11 illustrates an example environment 1100 for communicating atranscription and corresponding audio over a same communication channel.The environment 1100 may be arranged in accordance with at least oneembodiment described in the present disclosure. The environment 1100 mayinclude a first signal processing system 1164 and a second signalprocessing system 1166.

The first signal processing system 1164 may be an example of the firstsignal processing system 1064 of FIG. 10 and the second signalprocessing system 1166 may be an example of the second signal processingsystem 1066 of FIG. 10 . The first signal processing system 1164 and thesecond signal processing system 1166 may be communicatively coupled viaany suitable network (not expressly illustrated). For example, the firstsignal processing system 1164 and the second signal processing system1166 may be communicatively coupled via a phone line 1168 of a voicenetwork, which may be an example of the communication channel 1068 ofFIG. 10 . In some embodiments, the voice network may be an analog voicenetwork, a digital voice network, (e.g., a VoIP network) or any suitablecombination thereof. In the example of FIG. 11 , the arrows and linesillustrated as representing the phone line 1168 are merely to help withvisualizing that the phone line 1168 is between the first signalprocessing system 1164 and the second signal processing system 1166. Thearrows and lines are not meant to represent the actual path of the phoneline 1168.

In general, the first signal processing system 1164 may be configured tomultiplex audio 1162 and a transcription 1160 to generate combined data1150. Although referred to as “data,” reference to the combined data1150 may also refer to any suitable signal that may be used to carry theinformation that may be included in the combined data 1150. The audio1162 may be analogous to the audio 1062 of FIG. 10 . In addition, thetranscription 1160 may be analogous to the transcription 1060 of FIG. 10. As indicated above, reference to the audio 1162 or the transcription1160 may also refer to any suitable signal that may be used to carry theinformation associated therewith. As discussed in further detail below,the multiplexing of the audio 1162 and the transcription 1160 may besuch that the audio 1162 and the transcription 1160 may be communicatedtogether as the combined data 1150 over the phone line 1168 forreception by the second signal processing system 1166.

The first signal processing system 1164 may include a first audioprocessing system 1170 in some embodiments. The first audio processingsystem 1170 may include any suitable hardware and/or software configuredto process the audio 1162. In the present disclosure any operation thatmay be performed by the first audio processing system 1170 with respectto the audio 1162 may be considered as “processing” the audio 1162 toobtain “processed” audio 1162, which may be analogous to the processedaudio 1062 discussed above with respect to FIG. 10 . Such operations maybe referred to as “first audio processing operations” and may includeone or more operations that may include analysis operations, encodingoperations, filtering operations, data compression operations, frequencyand/or time multiplexing operations, signal relaying operations, signalrouting operations, bandwidth compression operations, frequencyshifting, phase shifting, signal storage, delay, multipliers,amplification, data and/or signal compression, speech enhancement, noisereduction, quantization, smoothing, interpolation, table lookups, linearor non-linear transformation, rectification, normalization, etc., or anysuitable combination thereof. For example, the first audio processingsystem 1170 may include one or more switches, encoders, analog filters,digital filters, multiplexers, digital signal processing systems, neuralnetworks, signal routers, etc., or any suitable combination thereofconfigured to perform one or more of the first audio processingoperations.

In these or other embodiments, the first signal processing system 1164may include a first transcription processing system 1172. The firsttranscription processing system 1172 may include any suitable hardwareand/or software configured to process the transcription 1160. In thepresent disclosure any operation that may be performed by the firsttranscription processing system 1172 with respect to the transcription1160 may be considered as “processing” the transcription 1160 to obtaina “processed” transcription 1160, which may be analogous to theprocessed transcription 1060 discussed above with respect to FIG. 10 .Such operations may be referred to as “first transcription processingoperations” and may include one or more operations that may includeanalysis operations, encoding operations, modulating operations,filtering operations, data compression operations, frequency and/or timemultiplexing operations, signal relaying operations, signal routingoperations, frequency shifting, phase shifting, signal storage, delay,multiplication, amplification, data and/or signal de-compression, speechenhancement, quantization, smoothing, interpolation, table lookups,linear or non-linear transformation, rectification, normalization, etc.,or any suitable combination thereof. For example, the firsttranscription processing system 1172 may include one or more switches,modems, encoders, analog filters, digital filters, multiplexers, digitalsignal processing systems, neural networks, signal routers, etc., or anysuitable combination thereof configured to perform one or more of thefirst transcription processing operations.

In these or other embodiments, the first signal processing system 1164may include a first filtering system 1174. The first filtering system1174 may include any suitable hardware and/or software configured toperform filtering operations with respect to the audio 1162 and/or thetranscription 1160. For example, the first filtering system 1174 mayinclude any suitable analog and/or digital filter configured to performthe filtering. For instance, the first filtering system 1174 may includeone or more analog components configured as any suitable filter.Additionally or alternatively, the first filtering system 1174 mayinclude a digital signal processing system that includes one or moredigital filters implemented in software and configured to perform anysuitable filtering operation. In these or other embodiments, the firstfiltering system 1174 may include one or more neural networks configuredto perform the filtering.

In the illustrated example of FIG. 11 , the first filtering system 1174may include a first filter 1176 and a second filter 1178. As discussedin further detail below, in some embodiments, the first filter 1176 mayinclude any suitable analog or digital filter configured to perform oneor more filtering operations with respect to the audio 1162.Additionally or alternatively, the second filter 1178 may include anysuitable analog or digital filter configured to perform one or morefiltering operations with respect to the transcription 1160. In these orother embodiments, the first filter 1176 and/or the second filter 1178may include any number of filters. Example filters that may be used asthe first filter 1176 and/or the second filter 1178 may include passbandfilters, band reject filters (e.g., notch filters), comb filters,filters with multiple passbands and/or reject bands, etc.

In some embodiments, and as explained in further detail below, thesecond filter 1178 may be the inverse of the first filter 1176. Forexample, the first filter 1176 may be configured to attenuatefrequencies that the second filter 1178 is configured to allow to passwith little to no attenuation. Similarly, the second filter 1178 may beconfigured to attenuate frequencies that the first filter 1176 isconfigured to allow to pass with little to no attenuation.

Further, although illustrated and depicted as being separate elements,the first audio processing system 1170, the first transcriptionprocessing system 1172, and the first filtering system 1174 may beimplemented in any suitable manner. For example, in some embodiments, atleast a portion of the first filtering system 1174 may be included inthe first audio processing system 1170 and/or the first transcriptionprocessing system 1172. For instance, in some embodiments, the firstfilter 1176 may be part of the first audio processing system 1170 andthe second filter 1178 may be part of the first transcription processingsystem 1172. Additionally or alternatively, in some embodiments thefirst audio processing system 1170 and the first transcriptionprocessing system 1172 may be combined. Further, in some embodiments,one or more elements may be omitted from the first signal processingsystem 1164. For example, in some embodiments, the first filteringsystem 1174 or one or more elements included therein may be omitted. Forinstance, in some embodiments, the first filter 1176 and/or the secondfilter 1178 may be omitted.

In general, the second signal processing system 1166 may be configuredto receive and process the combined data 1150 to reproduce the audio1162 and the transcription 1160 from the combined data 1150 to obtainreproduced data 1190. For example, the second signal processing system1166 may be configured to distinguish between and identify the audio1162 and the transcription 1160 included in the combined data 1150 toobtain the reproduced data 1190. Similar to as discussed above withrespect to FIG. 10 , in some instances, reference to the audio 1162 orthe transcription 1160 may include instances in which the audio 1162 orthe transcription 1160 may be considered processed audio 1162 and theprocessed transcription 1160. For example, as discussed in detail below,the audio 1062 and the transcription 1060 may be multiplexed intocombined data. As such, reference of the audio 1062 and thetranscription 1060 with respect to the combined data may also bereferring to the processed audio 1062 and the processed transcription1060 even if not explicitly stated as such.

In some embodiments, the second signal processing system 1166 mayinclude a second filtering system 1184. The second filtering system 1184may include any suitable hardware and/or software configured to performfiltering operations with respect to the audio 1162 and/or thetranscription 1160 included in the combined data 1150. For example, thesecond filtering system 1184 may include any suitable analog and/ordigital filter configured to perform the filtering. For instance, thesecond filtering system 1184 may include one or more analog componentsconfigured to as any suitable filter. Additionally or alternatively, thesecond filtering system 1184 may include a digital signal processingsystem that includes one or more digital filters implemented in softwareand configured to perform any suitable filtering operation. In these orother embodiments, the second filtering system 1184 may include one ormore neural networks configured to perform the filtering. In general,the filtering operations performed by the second filtering system 1184may be used to separate the audio 1162 from the transcription 1160 inthe combined data 1150.

In the illustrated example of FIG. 11 , the second filtering system 1184may include a first filter 1186 and a second filter 1188. As discussedin further detail below, in some embodiments, the first filter 1186 mayinclude any suitable analog or digital filter configured to perform oneor more filtering operations with respect to the audio 1162 included inthe combined data 1150. Additionally or alternatively, the second filter1178 may include any suitable analog or digital filter configured toperform one or more filtering operations with respect to thetranscription 1160 included in the combined data 1150. In these or otherembodiments, the first filter 1186 and/or the second filter 1188 mayinclude any number of filters. Example filters that may be used as thefirst filter 1186 and/or the second filter 1188 may include passbandfilters, band reject filters (e.g., notch filters), comb filters,filters with multiple passbands and/or reject bands, etc.

In some embodiments, and as explained in further detail below, thesecond filter 1188 may be the inverse of the first filter 1186. Forexample, the first filter 1186 may be configured to attenuatefrequencies that the second filter 1188 is configured to allow to passwith little to no attenuation. Similarly, the second filter 1188 may beconfigured to attenuate frequencies that the first filter 1186 isconfigured to allow to pass with little to no attenuation.

In these or other embodiments, the second signal processing system 1166may include a second audio processing system 1180. The second audioprocessing system 1180 may include any suitable hardware and/or softwareconfigured to process the combined data 1150 to reproduce the audio 1162from the combined data 1150. In the present disclosure any operationthat may be performed by the second audio processing system 1180 withrespect to the combined data 1150 to reproduce the audio 1162 from thecombined data 1150 may be referred to as “second audio processingoperations” and may include one or more operations that may includedecoding operations, demodulating operations, filtering operations, datade-compression operations, frequency and/or time de-multiplexingoperations, signal relaying operations, signal routing operations,bandwidth extension operations, frequency shifting, phase shifting,signal storage, delay, multiplication, amplification, data and/or signalde-compression, speech enhancement, noise reduction, quantization,smoothing, interpolation, table lookups, linear or non-lineartransformation, rectification, normalization etc., or any suitablecombination thereof. For example, the second audio processing system1180 may include one or more switches, decoders, analog filters, digitalfilters, multiplexers, digital signal processing systems, neuralnetworks, signal routers, modems, etc., or any suitable combinationthereof configured to perform one or more of the second audio processingoperations.

In these or other embodiments, the second signal processing system 1166may include a second transcription processing system 1182. The secondtranscription processing system 1182 may include any suitable hardwareand/or software configured to process the combined data 1150 toreproduce the transcription 1160 from the combined data 1150. In thepresent disclosure any operation that may be performed by the secondtranscription processing system 1182 with respect to the combined data1150 to reproduce the transcription 1160 from the combined data 1150 maybe referred to as “second transcription processing operations” and mayinclude one or more operations that may include decoding operations,demodulating operations, filtering operations, data de-compressionoperations, frequency and/or time de-multiplexing operations, signalrelaying operations, signal routing operations, bandwidth extensionoperations, frequency shifting, phase shifting, signal storage, delay,multiplication, amplification, data and/or signal de-compression, speechenhancement, noise reduction, quantization, smoothing, interpolation,table lookups, linear or non-linear transformation, rectification,normalization, etc., or any suitable combination thereof. For example,the second transcription processing system 1182 may include one or moreswitches, decoders, analog filters, digital filters, multiplexers,digital signal processing systems, neural networks, signal routers,modems, etc., or any suitable combination thereof configured to performone or more of the second transcription processing operations.

Although illustrated and depicted as being separate elements, the secondaudio processing system 1180, the second transcription processing system1182, and the second filtering system 1184 may be implemented in anysuitable manner. For example, in some embodiments, at least a portion ofthe second filtering system 1184 may be included in the second audioprocessing system 1180 and/or the second transcription processing system1182. For instance, in some embodiments, the first filter 1186 may bepart of the second audio processing system 1180 and the second filter1188 may be part of the second transcription processing system 1182.Additionally or alternatively, in some embodiments the second audioprocessing system 1180 and the second transcription processing system1182 may be combined. Further, in some embodiments, one or more elementsmay be omitted from the second signal processing system 1166. Forexample, in some embodiments, the second filtering system 1184 or one ormore elements included therein may be omitted. For instance, in someembodiments, the first filter 1186 and/or the second filter 1188 may beomitted.

Below are some examples of operations that may be performed by the firstsignal processing system 1164 and the second signal processing system1166 to generate the combined data 1150. In some embodiments, theoperations may be such that the audio 1162 and the transcription 1160utilize different communication resources of the phone line 1168 (e.g.,time periods and frequencies of the phone line 1168). For example, theoperations may be such that the audio 1162 and the transcription 1160are communicated at different times and/or over different frequencies.In these or other embodiments, the operations may be such that the audio1162 and the transcription 1160 are communicated using a samecommunication resource (e.g. at the same time and/or using the samefrequencies).

In some embodiments, the first signal processing system 1164 may beconfigured to generate the combined data 1150 by frequency multiplexingthe audio 1162 and the transcription 1160 by communicating thetranscription 1160 using audio frequency bands that may be differentfrom those of the audio 1162. For example, the first transcriptionprocessing system 1172 may be configured to process the transcription1160 into a processed transcription 1160 that may be an audio datasignal. For instance, the first transcription processing system 1172 maybe a modem configured to modulate the transcription 1160 onto a carrierwave. Additionally or alternatively, the modem may use a neural networkto convert the transcription 1160 into an audio data signal. In these orother embodiments, the first transcription processing system 1172 may beconfigured to communicate the transcription 1160 as audio tones usingDTMF signaling. In these or other embodiments, the first transcriptionprocessing system 1172 may be configured to process the transcription1160 such that the frequency or frequencies of the processedtranscription 1160 are at the edge of the frequency range that istypically part of audio communicated during communication sessions(e.g., the frequency range of human speech).

For instance, the frequency range of human speech may typically bebetween 100 Hertz (Hz) and 3600 Hz. In some embodiments based on thisrange, the first signal processing system 1064 may be configured toprocess the transcription 1160 such that the processed transcriptionutilizes frequencies that are less than 100 Hz and/or greater than 3600Hz. For instance, the first transcription processing system 1172 may beconfigured to modulate the transcription 1160 onto a carrier wave thatis greater than 3600 Hz to generate the processed transcription.Additionally or alternatively, the audio tones that may be used as audiodata signals may have frequencies that are greater than 3600 Hz and/orless than 100 Hz.

In these or other embodiments, the first signal processing system 1164may be configured to multiplex the audio 1162 and the transcription 1160to generate the combined data 1150 based on the frequency ranges thatcorrespond to the audio 1162 and the transcription 1160. For example, insome embodiments, the first audio processing system 1170 may beconfigured to relay, as processed audio, the audio 1162 to the phoneline 1168 for communication over the phone line 1168. Additionally oralternatively, the first transcription processing system 1172 maycommunicate, as the processed transcription included in the combineddata 1150, the transcription 1160 over the phone line 1168 using thecarrier wave that is greater than 3600 Hz.

In these or other embodiments, the first signal processing system 1164may be configured to relay the audio 1162 to the first filter 1176 ofthe first filtering system 1174. The first filter 1176 may be configuredto pass frequencies that correspond to the audio 1162 and to attenuatefrequencies that do not correspond to the audio 1162. For example, asindicated above, the frequencies that correspond to the audio 1162 maybe between 100 Hz and 3600 Hz. As such, in some embodiments, the firstfilter 1176 may be a lowpass filter configured to pass frequencies lessthan 3600 Hz and to attenuate frequencies greater than 3600 Hz. Asanother example, the first filter 1176 may be a bandpass filterconfigured to pass frequencies between 100 Hz and 3600 Hz and toattenuate frequencies outside of that range.

Additionally or alternatively, the first transcription processing system1172 may be configured to communicate the processed transcription 1160(e.g., the carrier wave that is greater than 3600 Hz having thetranscription 1160 modulated thereon) to the second filter 1178. Inthese or other embodiments, the second filter 1178 may be configuredbased on the frequencies associated with the processed transcription1160. For example, the second filter 1178 may be a highpass filterconfigured to attenuate frequencies lower than 3600 Hz and to passfrequencies higher than 3600 Hz. As another example, the second filter1178 may be a notch filter configured to attenuate frequencies that arebetween 100 Hz and 3600 Hz and to pass frequencies that are outside ofthat range.

In these or other embodiments, the second signal processing system 1166may be configured to receive the combined data 1150 and distinguish andidentify the transcription 1160 and the audio 1162 based on thefrequency bands used to communicate the processed audio and theprocessed transcription included in the combined data 1150. For example,the second signal processing system 1166 may include the secondfiltering system 1184 in some embodiments. Further, the first filter1186 may be analogous to the first filter 1176 of the first filteringsystem 1174 and may be configured to receive the combined data 1150 andto attenuate frequencies higher than 3600 Hz. As such, the first filter1186 may filter out the transcription 1160 from the combined data 1150while leaving the audio 1162 such that the audio 1162 is identified fromthe combined data 1150 as part of the reproduced data 1190. In these orother embodiments, the second audio processing system 1180 of the secondsignal processing system 1166 may be configured to communicate theidentified audio 1162 to any suitable audio system (e.g., the audiosystem 1012 of FIG. 10 ) for presentation.

Additionally or alternatively, the second filter 1188 of the secondfiltering system 1184 may be analogous to the second filter 1178 of thefirst filtering system 1174 and may be configured to receive thecombined data 1150 and to attenuate frequencies lower than 3600 Hz. Assuch, the first filter 1186 may filter out the audio 1162 from thecombined data 1150 while leaving the transcription 1160 (e.g., asmodulated on the carrier wave as the processed transcription orcommunicated as an audio data signal using frequencies higher than 3600Hz) to identify the transcription 1160 (e.g., as the processedtranscription 1160) from the combined data 1150. In these or otherembodiments, the second transcription processing system 1182 may beconfigured to demodulate the processed transcription 1160 filtered fromthe combined data 1150 to reproduce the transcription 1160 as part ofthe reproduced data 1190. In these or other embodiments, the secondtranscription processing system 1182 of the second signal processingsystem 1166 may be configured to communicate the demodulatedtranscription 1160 to any suitable transcription presentation system(e.g., the transcription presentation system 1014 of FIG. 10 ) forpresentation. The frequencies given in the above example are merelyexamples and are not meant to be limiting.

Additionally or alternatively, the first audio processing system 1170may be configured to perform one or more of any suitable compressionoperations with respect to the audio 1162 such that the audio 1162 maybe encoded as a compressed data signal included as part of the combineddata 1150. Examples of audio encoding methods that may be suitable forcompressing an audio signal include A-law, mu-law (a.k.a. G.711), AMR,G.722, G.722.1, G.723, G.726, G.728, G.729, GSM, MP3, Code ExcitedLinear Prediction, Speex, Opus, and FLAC, In these or other embodiments,the second audio processing system 1180 may be configured to decode(e.g., decompress) the audio 1162 that has been compressed by the firstaudio processing system 1170 to reproduce the audio 1162 as part of thereproduced data 1190.

In these or other embodiments, the first transcription processing system1172 may be configured to perform one or more of any suitablecompression operations with respect to the transcription 1160 such thatthe transcription 1160 may be encoded as a compressed data signalincluded as part of the combined data 1150. Examples of data encodingmethods that may be suitable for compressing a data signal includeHuffman coding, adaptive Huffman coding, pkzip, grammar-based codes,Lempel-Ziv-Welch (LZW) encoding, and arithmetic coding based on afinite-state machine. In these or other embodiments, the secondtranscription processing system 1182 may be configured to decode (e.g.,decompress) the transcription 1160 that has been compressed by the firsttranscription processing system 1172 to reproduce the transcription 1160as part of the reproduced data 1190.

In these or other embodiments, the compressed audio may use lessbandwidth than the uncompressed audio with respect to communication ofthe audio 1162 over the phone line 1168. The reduction in bandwidth mayleave more bandwidth available for communication of the transcription1160 over the phone line 1168. Additionally or alternatively, thecompressed transcription may use less bandwidth than the uncompressedtranscription with respect to communication of the transcription 1160over the phone line 1168. The reduction in bandwidth may be such thatless bandwidth may be used for communication of the transcription 1160over the phone line 1168 In these or other embodiments, the firsttranscription processing system 1172, the first filter 1176, the secondfilter 1178, the first filter 1186, the second filter 1188, and/or thesecond transcription processing system 1182 may be configured accordingto the reduction in bandwidth.

For example, the bandwidth of the compressed audio may be between 1000Hz and 2500 Hz as opposed to between 100 Hz and 3600 Hz. As such, thefirst transcription processing system 1172 may be configured to modulatethe transcription 1160 onto one or more carrier waves that are less than1000 Hz and/or greater than 2500 Hz. Conversely, the secondtranscription processing system 1182 may be configured to demodulate thetranscription 1160 from the corresponding carrier waves having thecorresponding frequencies. Additionally or alternatively, the firstfilter 1176 and the first filter 1186 may be configured to passfrequencies that are greater than 1000 Hz and/or less than 2500 Hz andto attenuate frequencies that are less than 1000 Hz and/or greater than2500 Hz. In these or other embodiments, the second filter 1178 and thesecond filter 1188 may be configured to attenuate frequencies that aregreater than 1000 Hz and/or less than 2500 Hz and to pass frequenciesthat are less than 1000 Hz and/or greater than 2500 Hz.

In some embodiments, the audio 1162 may be compressed using a filter(e.g., the first filter 1176) that removes or attenuates certainfrequency bands, such as frequencies over 3600 Hz. Additionally oralternatively, the audio 1162 may be compressed using one or more speechencoding methods such as CELP or MP3 that remove redundancy from thespeech signal or save bandwidth by removing information that isrelatively less important.

For example, the first audio processing system 1170 and second audioprocessing system 1180 may include a compression and restoration systemconfigured as an autoencoder to compress the audio 1162 for transmissionover the phone line 1168 and then restore the audio 1162. Theautoencoder may include a first neural network, followed by abottleneck, followed by a second neural network. The first neuralnetwork may include a number of input nodes that is greater than thenumber of output nodes and/or it may output a smaller number of samplesthan it receives. The first audio processing system 1170 may include thefirst neural network and may compress the audio 1162, which may includea speech signal, into a compressed representation for transmission overthe phone line 1168. The second audio processing system 1180 may includea second neural network. The second neural network may include a numberof input nodes that is less than the number of output nodes and/or itmay output a greater number of output samples than it receives. Thesecond neural network may use random signals for one or more of itsinputs. The second neural network may convert the compressedrepresentation back as an approximation of the audio 1162.

In some embodiments, the autoencoder may be trained by selecting weightsthat minimize the difference between the input of the first neuralnetwork and the output of the second neural network. Additionally oralternatively, the first signal processing system 1164 may input thetranscription 1160 as input to the first neural network and the secondsignal processing system 1166 may extract a reproduced transcription1160 from the output of the second neural network. In some embodiments,the second neural network may include a modem. In some embodiments, thefirst signal processing system 1164 may process the audio 1162 andtranscription 1160 so that they occupy overlapping frequency bandsand/or time slots when they are communicated over the phone line 1168and the two signals are then separated by the second signal processingsystem 1166. By processing the audio 1162 and the transcription 1160 insuch a manner, the same communication resources may be used tocommunicate both the audio 1162 and the transcription 1160. In these orother embodiments, the encoding of the audio 1162 and the transcription1160 together to use overlapping frequency bands and/or time slots maybe such that the communication resources used (e.g., the frequency bandsand time slots) for the resulting combined data 1150 may be the same asthose that may have been used to communicate only the audio 1162 or onlythe transcription 1160. Therefore, the processing in which the audio1162 and the transcription 1160 of the combined data 1150 use the samecommunication resources may free up communication resources and/or allowfor the communication of both the audio 1162 and the transcription 1160in instances in which the communication resources were typically onlysufficient to communicate one or the other.

In some embodiments, an encoding system 1264 and decoding system 1266described below in relation to FIGS. 12A and 12B may be configured as anautoencoder and trained using a Generative Adversarial Network (GAN), asdescribed in relation to FIG. 12A in detail below.

Additionally or alternatively, the first audio processing system 1170may be configured to perform one or more of any other suitable bandwidthlimiting operations with respect to the audio 1162 such that the audio1162 as communicated over the phone line 1168 may use less bandwidth. Inthese or other embodiments, the second audio processing system 1180 maybe configured to decode (e.g., restore) the audio 1162 using anysuitable bandwidth extension or voice enhancement operations toreproduce the audio 1162 as part of the reproduced data 1190. Forexample, in some embodiments, a neural network (e.g., a GAN, a deepneural network, a recurrent neural network such as WaveNet, etc.) may beused to restore the audio 1162. Methods for restoring the audio includespeech enhancement and bandwidth extension methods. For example, thefirst filter or first audio processing system may band-limit the audioto, for example, 100 Hz to 3000 Hz, leaving the 3 kHz-4 kHz bandavailable for transmitting. The second audio processing system 1180 maythen use a bandwidth extension method to restore the removed portion.For example, a DNN (e.g., RNN, LSTM, GAN, WaveNet, etc.) may take theportion of the audio that was not removed (100 Hz to 3000 Hz, in theexample above) and use it to generate an estimate of the signal that wasremoved (3 kHz-4 kHz in the example). The generated estimate (3-4 kHz)may be added to the portion of the signal not removed (100 Hz-3 kHz) toform a reconstruction of the audio 1162 in the original state. In theseor other embodiments, similar bandwidth reduction and restorationoperations may be performed with respect to the transcription 1160 bythe first transcription processing system 1172 and the secondtranscription processing system 1182, respectively. In some embodiments,the second signal processing system 1166 may use bandwidth extension toextend the bandwidth of the audio 1162 beyond its original frequencyspan. For example, if audio 1162 is obtained from a telephone networkthat limits the highest frequency to below 4 kHz, bandwidth extensionmay be used to generate a representation of audio 1162 with an audiobandwidth up to 8 kHz for playback by the presentation system 1006 to alistener or as input to a speech recognizer.

Additionally or alternatively, the first audio processing system 1170may be configured to analyze (e.g., track) the audio 1162 to identifywhich frequencies are currently being used by the audio 1162. The firstaudio processing system 1170 may be configured to analyze whichfrequencies are currently being used using any suitable technique. Forexample, the first audio processing system 1170 may be configured todetect energy levels associated with the frequencies of the frequencyspectrum that may be used by the audio 1162. In these or otherembodiments, the first audio processing system 1170 may notify the firsttranscription processing system 1172 such that the first transcriptionprocessing system 1172 may modulate the transcription 1160 onto one ormore carrier waves that are outside of the currently used frequencies.

For example, during silence, the entire frequency spectrum that may beused to communicate the audio 1162 may be used to communicate thetranscription 1160 because the audio 1162 may not be using any of thefrequencies at that time. As another example, when a speaker is makingan “mmm” sound, frequency bands above 2000 Hz that are associated withthe audio 1162 may have little to no energy such that those bands may beused to communicate the transcription 1160. As such, the first signalprocessing system 1164 may multiplex the audio 1162 and thetranscription 1160 by analyzing and relaying the audio 1162 andmodulating the transcription 1160 using frequencies not currently beingused by the audio 1162.

In these or other embodiments, the first audio processing system 1170may notify the first filtering system 1174 of the currently usedfrequencies such that the filtering frequencies of the first filter 1176and/or the second filterer 1178 may be adjusted accordingly.Additionally or alternatively, in some embodiments, the currently usedfrequencies may be communicated to the second signal processing system1166 such that the second transcription processing system 1182 and/orthe second filtering system 1184 may be adjusted to be able todistinguish between and identify the audio 1162 and the transcription1160. In some embodiments, the communication of the currently usedfrequencies may be performed using a same frequency band to enable thesecond signal processing system 1166 to obtain the information relatedto the currently used frequencies.

Additionally or alternatively, the second audio processing system 1180may be configured to analyze (e.g., track) the audio 1162 of thecombined data 1150 to identify the currently used frequencies to helpidentify which frequencies may be associated with the transcription 1160of the combined data 1150. In these or other embodiments, the secondaudio processing system 1180 may communicate such information to thesecond transcription processing system 1182 and/or the second filteringsystem 1184 such that the information may be used to identify andreproduce the transcription 1160 from the combined data 1150.

Additionally or alternatively, the first signal processing system 1164may be configured to time multiplex the audio 1162 and the transcription1160 to generate the combined data 1150. For example, the first audioprocessing system 1170 may be configured to analyze the audio 1162 toidentify points of time at which little to no audio is being sent (e.g.,to identify pauses in the conversation, silence, etc.). In these orother embodiments, the first audio processing system 1170 may beconfigured to indicate to the first transcription processing system 1172to modulate and communicate the transcription 1160 over the phone line1168 during the pauses or silence. In some embodiments, the audio 1162and/or the transcription 1160 as modulated may be communicated throughthe first filtering system 1174 prior to being communicated via thephone line 1168. Additionally or alternatively, the first filteringsystem 1174 may be bypassed by the audio 1162 and/or the transcription1160.

Additionally or alternatively, in some embodiments, the time periodsover which the transcription 1160 may be communicated over the phoneline 1168 may be communicated to the second signal processing system1166 such that the second transcription processing system 1182 and/orthe second filtering system 1184 may be adjusted to be able todistinguish between and identify the audio 1162 and the transcription1160. In some embodiments, the communication of the time periods may beperformed using a same frequency band to enable the second signalprocessing system 1166 to obtain the information related to the timeperiods.

In these or other embodiments, the first audio processing system 1170may be configured to perform one or more time compression operations onthe audio 1162 such that the audio 1162 may be communicated over smallerperiods of time. For example, the first audio processing system 1170 maybe configured to speed up the audio 1162 (e.g., by increasing the speechrate or shortening or eliminating silences) to increase the duration ofthe time periods that may not be used by the audio 1162 and thatconsequently may be used to communicate the transcription 1160. In theseor other embodiments, the second audio processing system 1180 may beconfigured to perform a complementary process on the sped-up audio 1162to reproduce the audio 1162. For example, the second audio processingsystem 1180 may be configured to slow down and/or repair the sped-upaudio 1162 using any suitable process that may complement that used tospeed up the audio 1162.

Additionally or alternatively, the first transcription processing system1172 may be configured to analyze the transcription 1160 to determine anamount of data included in the transcription 1160. In these or otherembodiments, the first audio processing system 1170 may process theaudio 1162 based on the amount of data included in the transcription1160. For example, in response to the amount of data included in thetranscription being relatively high (e.g., as determined by beinggreater than a high data threshold), the first audio processing system1170 may adjust one or more of the bandwidth, speed, compression, etc.,as discussed above to render more communication channel resources (e.g.,frequency, time) available for communicating the transcription 1160 overthe phone line 1168. As another example, in response to the amount ofdata included in the transcription being relatively low (e.g., asdetermined by being less than a low data threshold), the first audioprocessing system 1170 may adjust one or more of the bandwidth, speed,compression, etc., as discussed above to render more communicationchannel resources (e.g., frequency, time) available for communicatingthe audio 1162 over the phone line 1168.

Additionally or alternatively, the audio 1162 may be prioritized overthe transcription 1160 for any other suitable reason or vice versa andthe first audio processing system 1170 and the first transcriptionprocessing system 1172 may be configured to operate according to thecurrent prioritization. In these or other embodiments, in instances inwhich the audio 1162 is prioritized over the transcription 1160, thefirst transcription processing system 1172 may be configured to bufferthe transcription 1160 (e.g., using any suitable storage buffer such asshift registers, FIFO (first in first out) registers, blocks of memory,etc.) until more communication resources are available for communicationof the transcription 1160 in instances in which the amount of dataincluded in the transcription 1160 is more than what may be communicatedover the communication resources allocated for communication of thetranscription 1160.

In some embodiments, the first signal processing system 1164 may includean encoding system that includes one or more encoders that areconfigured to perform processing on the audio 1162 and/or thetranscription 1160 to encode the audio 1162 and/or the transcription1160 for communication as the combined data 1150. The encoding mayinclude combining the audio 1162 and the transcription 1160, frequencyshifting and/or frequency compressing of the audio 1162 and/or thetranscription 1160, compressing the audio 1162 and/or the transcription1160, time shifting and/or time compressing the audio 1162 and/or thetranscription 1160 (e.g., speeding up the audio 1162 as discussedabove), attenuating certain frequencies of the audio 1162 and/or thetranscription 1160 (e.g., filtering the audio 1162 and/or thetranscription 1160), amplifying the audio 1162 and/or the transcription1160, amplifying certain frequencies of the audio 1162 and/or thetranscription 1160, or any suitable combination thereof.

In these or other embodiments, all or portions of the first audioprocessing system 1170, the first transcription processing system 1172,and/or the first filtering system 1174 may be implemented as the one ormore encoders or may include the one or more encoders of the encodingsystem. Additionally or alternatively, the one or more encoders may beused to perform one or more of the frequency multiplexing or timemultiplexing operations described above. In these or other embodiments,the one or more encoders may include or may be implemented as one ormore first neural networks. The one or more first neural networks mayinclude any suitable neural network including a deep neural network(DNN), a GAN, or any other suitable neural network, or combinationthereof. Further, “a neural network” in the present disclosure mayinclude any number of neural networks such that reference to “a neuralnetwork” or “the neural network” is not limited to a single neuralnetwork.

Additionally or alternatively, the second signal processing system 1166may include a decoding system that includes one or more decoders thatare configured to perform processing on the combined data 1150 to decodethe combined data and reproduce the audio 1162 and the transcription1160 as the reproduced data 1190. The decoding may include any suitableoperation that may be complementary to the encoding operations toreverse the encoding in a manner that allows for reproducing the audio1162 and the transcription 1160. For example, the decoding operationsmay include separating the audio 1162 and the transcription 1160,frequency shifting and/or frequency expanding the audio 1162 and/or thetranscription 1160, decompressing the audio 1162 and/or thetranscription 1160, time shifting and/or time decompressing the audio1162 and/or the transcription 1160 1160 (e.g., slowing down the sped upaudio 1162 as discussed above), attenuating certain frequencies of theaudio 1162 and/or the transcription 1160 (e.g., filtering the audio 1162and/or the transcription 1160), amplifying the audio 1162 and/or thetranscription 1160, amplifying certain frequencies of the audio 1162and/or the transcription 1160, or any suitable combination thereof.

In these or other embodiments, all or portions of the second audioprocessing system 1180, the second transcription processing system 1182,and/or the second filtering system 1184 may be implemented as the one ormore decoders or may include the one or more decoders of the decodingsystem. Additionally or alternatively, the one or more decoders may beused to perform one or more of the distinguishing and identifyingoperations described above with respect to distinguishing andidentifying the audio 1162 and the transcription 1160 from the combineddata 1150 to reproduce the audio 1162 and the transcription 1160 as thereproduced data 1190. In these or other embodiments, the one or moredecoders may include or may be implemented as one or more second neuralnetworks. The one or more second neural networks may include anysuitable neural network including a deep neural network (DNN), a GAN, orany other suitable neural network, or combination thereof.

In some embodiments the first and second neural networks may havetrainable weights and or parameters that may be associated with thedifferent operations that may be performed in the encoding andcorresponding decoding of the combined data 1150 such that the first andsecond neural networks may be biased to performing certain operationsmore than other operations. The training of the first and second neuralnetworks may be performed according to any suitable technique and someexamples of which are discussed in further detail below with respect toFIG. 12A. In the present disclosure, reference to encoding of thecombined data 1150 by the first neural networks may include any encodingoperation that may be performed with respect to the audio 1162, thetranscription 1160, or a combination of the audio 1162 and thetranscription 1160 to obtain the combined data 1150. Similarly, in thepresent disclosure, reference to decoding of the combined data 1150 bythe first neural networks may include any encoding operation that may beperformed with respect to the audio 1162, the transcription 1160, or acombination of the audio 1162 and the transcription 1160 to obtain thecombined data 1150. Additionally, reference to training the encodingsystem or training the decoding system may refer to training of thefirst and second neural networks associated therewith.

In some embodiments, the encoding system and the decoding system may beconfigured to operate based on the conditions of the phone line 1168.For example, the phone line 1168 may support a 3.2 kHz bandwidthtelephone call, a 3.6 kHz telephone call, a landline phone call, acellular phone call, video calls, VoIP calls, etc. Further, in someinstances, the audio 1162 may be clear, muffled, noisy, distorted,contain artifacts (e.g., from compression), etc. Additionally oralternatively, the available bandwidth for communicating the combineddata 1150 may vary. In these or other embodiments, the encoding systemmay be configured to detect one or more of the variable conditions(e.g., to detect a channel type of the phone line 1168, a condition ofthe phone line 1168 (e.g., loss of the phone line 1168, noise on thephone line 1168, interference from other signals experienced on thephone line 1168, distortion created by the phone line 1168, signal loss,etc.) and may be configured to adjust the encoding to offset or reducenegative effects that may be experienced by the combined data 1150 thatmay be associated therewith. In these or other embodiments, the encodingsystem may include multiple encoders that are each configured for aspecific one of the monitored conditions and may select which encoder touse based on the detected conditions of the phone line 1168.

Below are some examples of how the neural networks may be implementedwith the first signal processing system 1164 and the second signalprocessing system 1166. However, the below examples are not meant to belimiting and the implementations may vary depending on different designand operational considerations.

In some embodiments, the first audio processing system 1170 may includea first DNN and a first “n” bit shift register. In these or otherembodiments, the first transcription processing system 1172 may includea modem. Additionally or alternatively, the first transcriptionprocessing system 1172 may modulate the transcription 1160 (e.g., viathe modem) and the second filter 1178 may filter the modulated signal tocreate the processed transcription 1160 as a filtered data signal. Inthese or other embodiments, digital samples of the audio 1162 may beobtained by the first shift register of the first audio processingsystem 1170. The first shift register may store the n most recent audiosamples, s₁, s, . . . , s_(n), and may provide the samples as input tothe first DNN. The first DNN may output, as the processed audio 1162, anencoded audio stream that is encoded using any suitable encodingoperation. For example, the encoding may include one or more bandwidthcompression operations, frequency compression operations, timecompression operations, data compression operations, linear or nonlineartransformations, etc., described above. Processed audio output from thefirst DNN may be fed back as input to the first DNN. The audio fed backto the first DNN may be provided in its original form and/or it may befiltered and/or delayed by one or more samples. The encoded audio streammay be communicated to the first filter 1176 and may be filtered by thefirst filter 1176 and then communicated over the phone line 1168 as partof the combined data 1150.

In some embodiments, the combined data 1150 may be received by thesecond filtering system 1184. The processed transcription 1160 may beseparated from the combined data 1150 using the second filter 1188. Inthese or other embodiments, the second transcription processing system1182 may be a modem that is configured to receive the data signal of theprocessed transcription 1160 from the second filter 1188 and demodulatethe data signal of the processed transcription 1160 to reproduce thetranscription 1160.

Additionally or alternatively, the second audio processing system 1180may include a second DNN and a second “n” bit shift register. In theseor other embodiments, the first filter 1186 of the second filteringsystem 1184 may be configured to separate the processed (e.g., encoded)audio 1162 from the combined data 1150 and to communicate the processedaudio 1162 to the second shift register. The second shift register maystore then most recent audio samples, s₁, s, . . . , s_(n), of theprocessed audio 1162 and may provide the samples as input to the secondDNN. The second DNN may have a structure similar to that of the firstDNN in that it inputs multiple audio samples and reads out a singlesample at a time, though it may have a different DNN topology. As withthe first DNN, the output of the second DNN may be processed and/or fedback to the input of the second DNN. The second DNN may be configured todecode the encoding done by the first DNN such that the output of thesecond DNN may be a reproduction of the audio 1162 as received by thefirst DNN.

Modifications may be made to the above example, without departing fromthe scope of the present disclosure. For example, the above example mayomit the first filter 1176 and/or the second filter 1178 of the firstfiltering system 1174 or the first filter 1186 and/or the second filter1188 of the second filtering system 1184.

As another example, at least a portion of the first audio processingsystem 1170 and the first transcription processing system 1172 may beimplemented as a first shift register, a second shift register, and afirst DNN. In these or other embodiments, the first transcriptionprocessing system 1172 may include a modem communicatively coupled tothe second shift register. Additionally, at least a portion of thesecond audio processing system 1180 may be implemented as a third shiftregister and a second DNN and the second transcription processing system1182 may be implemented as a fourth shift register and a third DNN. Inthese or other embodiments, the second transcription processing system1182 may include a modem communicatively coupled to the third DNN.

In such an example, the audio 1162 may be communicated to the firstshift register and the transcription 1160 may be modulated onto a datasignal (e.g., an audio data signal) by the modem, which may becommunicated to the second shift register. The outputs of the first andsecond shift registers may be received by the first DNN as an input,such that the first DNN may encode the audio 1162 with the transcription1160 to generate the combined data 1150. The combined data 1150 may becommunicated to the third shift register and the fourth shift register.The output of the third shift register may be communicated to the secondDNN of the second audio processing system 1180, which may be configuredto distinguish and identify the audio 1162 as encoded in the combineddata 1150 to reproduce the audio 1162 as part of the reproduced data1190. In some embodiments, the decoding operations performed by thesecond DNN to identify the audio 1162 from the combined data 1150 may bebased on the encoding of the first DNN such that the second DNN is ableto identify the audio 1162 from the combined data 1150.

In these or other embodiments, the output of the fourth shift registermay be communicated to the third DNN of the second transcriptionprocessing system 1182, which may be configured to distinguish andidentify the transcription 1160 as encoded in the combined data 1150. Insome embodiments, the decoding operations performed by the third DNN toidentify the transcription 1160 as encoded in the combined data 1150 maybe based on the encoding of the first DNN such that the third DNN isable to identify the transcription 1160 from the combined data 1150. Inthese or other embodiments, the separated transcription 1160 may becommunicated from the third DNN to the modem of the second transcriptionprocessing system 1182 to demodulate the corresponding signal toreproduce the transcription 1160 as part of the reproduced data 1190.

Modifications may be made to the above example, without departing fromthe scope of the present disclosure. For example, the above example mayinclude the first filter 1176 and/or the second filter 1178 of the firstfiltering system 1174 or the first filter 1186 and/or the second filter1188 of the second filtering system 1184. Additionally or alternatively,the modulating modem described may be omitted and the modulating of thetranscription 1160 may be performed by the first DNN. In these or otherembodiments, the demodulating modem may be omitted and the demodulatingof the transcription 1160 may be performed by the third DNN. In these orother embodiments, the second DNN and the third DNN may be combined intoa single DNN and/or the third shift register and the fourth shiftregister may be combined into a single shift register.

As another example, the neural networks of the encoding system and thedecoding system may be configured as GANs that may run in a generativemode. For example, a block of the 1162 and/or of the transcription 1160may be applied to the input of the encoding system. The encoding systemmay use recurrent neural network (RNN) layers such as Long Short TermMemory Layers (LSTMs). In these or other embodiments, the state of theencoding system (e.g., the value of at least some of the signals insidethe encoding system such as outputs of nodes or values of weightsapplied to connections between nodes included in a neural network of theencoding system) after the block of the audio 1162 or the block of thetranscription 1160 is processed may then be transmitted to the decodingsystem. At least part of the decoding system (e.g., at least part of aneural network of the decoding system) may be initialized based on thereceived state values and run using a random signal as input to decodethe corresponding data.

The environment 1100 may thus be used to communicate the transcription1160 and the audio 1162 together over the same phone line 1168. Suchcommunication may allow for the providing of transcription services ininstances in which the communication of the transcription 1160 may belimited to the same communication channels used to communicate the audio1162. In some embodiments, the transcription and audio may becommunicated at the same time and using the same frequency bands.

Modifications, additions, or omissions may be made to the environment1100 and/or the components operating in the environment 1100 withoutdeparting from the scope of the present disclosure. For example, in someembodiments, the environment 1100 may be integrated into otherenvironments that provide additional benefits for a user. As anotherexample, the particular arrangement and description of the componentsare merely examples used to help explain the concepts described hereinand are not meant to be limiting.

Further, the use of shift registers in providing data streams to aneural network is illustrative. Other methods of providing the data tothe neural networks and subsequent encoding and decoding may also beused. For example, the data (e.g., the audio 1162 and/or thetranscription 1160) may be applied to a neural network serially, via asingle input, and the neural network may store information based onmemory (using, for example, LSTMs or other RNNs) of past digital samplesof the data. The arrangements illustrated for extracting the data from aneural network are also illustrative. Other methods exist, includingconfiguring a neural network with multiple outputs, each representing adata sample and where the multiple outputs represent a segment of thecorresponding data (e.g., a segment of the audio 1162 or of thetranscription 1160). The number of nodes, number of layers, nature ofthe activation functions, and topology of the neural networks may alsovary. For example, topologies of neural networks that may be used mayinclude varying numbers of layers and nodes, and networks with recurrentlayers (RNNs), convolutional layers (CNNs), long short term memory(LSTM) layers, layers with gated recurrent units (GRU), residual neuralnetworks (ResNet), and generative networks such as GANs, WaveNet, andteacher/student training variations of WaveNet.

FIG. 12A illustrates an example environment 1200 for training anencoding system 1264 and a decoding system 1266. The environment 1200may be arranged in accordance with at least one embodiment described inthe present disclosure. The environment 1200 may include the encodingsystem 1264, the decoding system 1266, a network 1202, and a trainingsystem 1254.

The encoding system 1264 may include one or more encoders that areconfigured to perform processing on audio and/or a transcription (e.g.,the audio 1162 and/or the transcription 1160 of FIG. 11 ) to encode theaudio and/or the transcription for communication as combined data (e.g.,the combined data 1150 of FIG. 11 ). The encoding may include combiningthe audio and the transcription, frequency shifting and/or frequencycompressing of the audio and/or the transcription, compressing the audioand/or the transcription, time shifting and/or time compressing theaudio and/or the transcription (e.g., speeding up the audio),attenuating certain frequencies of the audio and/or the transcription(e.g., filtering the audio and/or the transcription), amplifying theaudio and/or the transcription, amplifying certain frequencies of theaudio and/or the transcription, combining the audio and transcriptionsto use overlapping communication resources (e.g., overlapping frequencybands and/or time slots), or any suitable combination thereof. Theencoding system 1264 may be an example of or part of the first signalprocessing system 1164, the first audio processing system 1170, thefirst transcription processing system 1172, and/or the first filteringsystem 1174 of FIG. 11 . In addition, the encoding system 1264 may be anexample of the encoding system described above with respect to FIG. 11 .

The decoding system 1266 may include one or more decoders that areconfigured to perform processing on the combined data to decode thecombined data and reproduce the audio and the transcription asreproduced data. The decoding may include any suitable operation thatmay be complementary to the encoding operations to reverse the encodingin a manner that allows for reproducing the audio and the transcription.For example, the decoding operations may include separating the audioand the transcription, frequency shifting and/or frequency expanding theaudio and/or the transcription, decompressing the audio and/or thetranscription, time shifting and/or time decompressing the audio and/orthe transcription (e.g., slowing down the sped up audio), attenuatingcertain frequencies of the audio and/or the transcription (e.g.,filtering the audio and/or the transcription), amplifying the audioand/or the transcription, amplifying certain frequencies of the audioand/or the transcription, separating combined audio and transcriptionsthat use overlapping communication resources, or any suitablecombination thereof. The decoding system 1266 may be an example of orpart of the second signal processing system 1166, the second audioprocessing system 1180, the second transcription processing system 1182,and/or the second filtering system 1184 of FIG. 11 . In addition, thedecoding system 1266 may be an example of the decoding system describedabove with respect to FIG. 11 .

In some embodiments, the network 1202 may be analogous to the network1002 of FIG. 10 . In the illustrated example of FIG. 12A, the network1202 may be configured to communicatively couple the encoding system1264 and the decoding system 1266. In some embodiments, the network 1202may be an analog network, such as an analog voice network. Additionallyor alternatively, the network 1202 may be an actual network or asimulated network configured to simulate the conditions of an actualnetwork. In these or other embodiments, the network 1202 may include acommunication channel 1268 that may be used to communicate information(e.g., audio and/or a transcription of a communication session). Thecommunication channel 1268 may be analogous to the communication channel1068 or 1168 of FIGS. 10 and 11 , respectively. Additionally oralternatively, the communication channel 1268 may be a simulation of acommunication channel of the network 1202 but may not be an actualcommunication channel. In the example of FIG. 12A, the arrows and linesillustrated as representing the communication channel 1268 are merely tohelp with visualizing that the communication channel 1268 is between theencoding system 1264 and the decoding system 1266. The arrows and linesare not meant to represent the actual path of the communication channel1268. For example, although the arrows and lines associated with thecommunication channel 1268 do not pass through the network 1202 in theillustration of FIG. 12A, the communication channel 1268 may be part ofand pass through the network 1202.

During training, training data 1248 may be provided to the encodingsystem 1264. The training data 1248 may include audio examples and/ortranscription examples that may be encoded by the encoding system 1264.For example, the training data 1248 may include recordings of audio ofalready conducted communication sessions. In these or other embodiments,the training data 1248 may include transcriptions of the alreadyconducted communication sessions. In these or other embodiments, thetraining data 1248 may include particular audio and a correspondingparticular transcription of a current communication session. Althoughreferred to as “data,” reference to the training data 1248 may alsorefer to any suitable signal that may be used to carry the informationthat may be included in the training data 1248.

In some embodiments, whether the training data 1248 includes audioexamples or transcription examples may depend on the type of data theencoding system 1264 is configured to encode. For example, in instancesin which the encoding system 1264 is configured to encode only audio,the training data 1248 may include audio examples but not transcriptionexamples. As another example, in instances in which the encoding system1264 is configured to encode audio and transcriptions, the training data1248 may include audio examples and transcription examples. As anotherexample, in instances in which the encoding system 1264 is configured toencode only transcriptions, the training data 1248 may includetranscription examples but not audio examples.

The encoding system 1264 may be configured to encode the training data1248 into encoded data 1250. Although referred to as “data,” referenceto the encoded data 1250 may also refer to any suitable signal that maybe used to carry the information that may be included in the encodeddata 1250. In some embodiments, the encoded data 1250 may thus includeaudio, a transcription, or a combination of audio and a correspondingtranscription. In some embodiments, the encoding system 1264 may beconfigured to filter the audio examples and/or transcription examples ofthe training data 1248 such as described above with respect to FIG. 11 .Additionally or alternatively, in some embodiments, the encoding system1264 may be configured to modulate the transcription examples of thetraining data 1248 in instances in which the training data includestranscription examples.

The encoding system 1264 may be configured to communicate the encodeddata to the decoding system 1266 via the communication channel 1268 ofthe network 1202. In instances in which the communication channel 1268is simulated, the communication of the encoded data 1250 may be to asuitable system configured to perform operations on the encoded data1250 as if the encoded data 1250 were communicated over the equivalentand actual communication channel 1268 of the network 1202. As theencoded data 1250 travels from the encoding system 1264 to the decodingsystem 1266 (or is simulated as traveling from the encoding system 1264to the decoding system 1266), the communication channel 1268 and thenetwork 1202 may impose a range of distortion, noise, compression,band-limiting, quantization, filtering, and other impairments on theencoded data 1250 such that the encoded data 1250 and accompanyingsignal may be changed as the encoded data propagates from the encodingsystem 1264 to the decoding system 1266.

In instances in which the network 1202 and the communication channel1268 are simulated, the network simulator that is used may represent therange and prevalence of audio impairments caused by the real network,including mu-Law encoding, noise, Analog to Digital (A/D) and Digital toAnalog (D/A) imperfections such as quantization, data or packet loss,amplitude variation, noise, bandwidth limits, signal compression,artifacts caused by signal compression, etc. These variations andimpairments may be imposed at random during training or they may beimposed sequentially so that the simulator cycles through a range ofchannel conditions. In these or other embodiments, the training may usebackpropagation, in which case the network simulator may be configuredto perform a set of mathematical functions that are included in thebackpropagation training process. If a real network is used fortraining, a representative set or range of network conditions may beused during the training.

The decoding system 1266 may receive the encoded data 1250 and maydecode the encoded data 1250 to generate decoded data 1290. Althoughreferred to as “data,” reference to the decoded data 1290 may also referto any suitable signal that may be used to carry the information thatmay be included in the decoded data 1290.

The decoded data 1290 may be a reproduction of the training data 1248but with a certain degree of distortion as compared to the training data1248. The distortion may be caused by encoding, decoding, and the changeof the encoded data 1250 that may be created as the encoded data 1250propagates (or is simulated as propagating) over the communicationchannel 1268. As indicated above, the encoding system 1264 and thedecoding system 1266 may include neural networks that have trainableweights and parameters that may bias which operations may be performedfor the encoding and the decoding. In some embodiments, the distortionmay be used to train the encoding system 1264 (e.g., by training thecorresponding neural networks) and to train the decoding system (e.g.,by training the corresponding neural networks) to set the weights andthe parameters such that the distortion may be reduced or minimized. Assuch, the encoding system 1264 and the decoding system 1266 may betrained to compensate for the distortion that may be caused as theencoded data 1250 propagates from the encoding system 1264 to thedecoding system 1266 such that the decoded data 1290 may be areproduction of or substantial reproduction of the training data 1248.

In some embodiments, the environment 1200 may include a training system1254 configured to determine the distortion between the decoded data1290 and training data 1248. The training system 1254 may include anysuitable hardware and/or software configured to perform the operationsdescribed herein with respect to the training system 1254.

In some embodiments, the training system 1254 may be configured toreceive the training data 1248 and the decoded data 1290 to determine anerror 1252 that may be a representation of the distortion between thedecoded data 1290 and the training data 1248. The error 1252 may includeany suitable representation of the distortion and may includeinformation that indicates the distortion and/or one or more signalsthat carry the information or represent the distortion.

In some embodiments, the training system 1254 may be configured to use aloss function that compares the training data 1248 to the decoded data1290 to generate the error 1252. For example, in some embodiments, theerror 1252 may be a simple subtraction of decoded data 1290 and thetraining data 1248. In some embodiments, the error 1252 may be thesquared difference between decoded data 1290 and the training data 1248.

Additionally or alternatively, the error 1252 may be generated by othermethods such as comparisons of frequency spectra of the training data1248 and the decoded data 1290 or using GANs. For example, with respectto audio of the training data 1248 and of the decoded data 1290, theaudio may be segmented into frames by extracting windows of audio thatmay be time periods of audio (e.g., 40 ms time periods). In someembodiments, the windows may overlap. For example a first window may be40 ms of audio and may start at time t and ending at time t+40. In theseor other embodiments, a second window may also be 40 ms of audio and maystart at time t+20 and ending at time t+60. In these or otherembodiments, a tapering function such as a raised cosine, Blackman, orHamming window may be applied to each window to reduce spectral leakage.Additionally or alternatively, A magnitude spectrum of each window maybe determined by transforming the corresponding audio signal to acomplex spectrum using a Fourier transform and taking the absolute valueto determine a magnitude spectrum. The magnitude spectrum may berepresented in various forms, including using cepstral coefficients,Mel-frequency cepstral coefficients (MFCCs), adding energy features, andadding delta- and delta-delta features. In these or other embodiments,the sum of absolute differences or squared differences may be determinedbetween the magnitude spectra of the decoded data 1290 and the trainingdata 1248 to produce the error 1252. In some embodiments, two or morefunctions may be combined, such as by determining a weighted sum of theabsolute magnitude spectrum difference and the squared differencebetween the corresponding time signal, to produce the error 1252. Theabove is meant as only an example of how the error 1252 may bedetermined using frequency spectra and is not meant to be limiting.

In instances in which the training data includes both audio examples andcorresponding transcription examples, the error 1252 may include acombination of a first error determined for the audio examples and asecond error determined for the corresponding transcription examples.The combination may be a weighted combination or unweighted combinationand may be a sum or product of the first error and the second error, orany other suitable combination. In some embodiments, the decoding system1266 may be configured to distinguish between and identify the audioexamples from the corresponding transcription examples using anysuitable technique such as described above.

In these or other embodiments, the training system 1254 may beconfigured to communicate the error 1252 to the encoding system 1264 andthe decoding system 1266. The error 1252 may be used by the encodingsystem 1264 and the decoding system 1266 to train the encoding system1264 and the decoding system 1266 to reduce or minimize the distortion(e.g., to reduce or minimize the error 1252). The neural networks of theencoding system 1264 and the decoding system 1266 may be trainedindividually or alternately or two or more may be trainedsimultaneously. In some embodiments, the encoding system 1264 and thedecoding system 1266 may be trained using any suitable cost functionthat may use the error 1252 as an input for training the encoding system1264 and the decoding system 166.

For example, in some embodiments, the cost function may include aniterative process in which a first instance of the training data 1248 isencoded and then decoded to determine a first instance of the error 1252and then one or more parameters and/or weights are adjusted. A secondinstance of the training data 1248 may then be encoded and decoded and asecond instance of the error 1252 may be determined and compared againstthe first instance of the error 1252 to determine whether the secondinstance indicates less distortion than the first instance. In responseto the second instance being less than the first instance, furtheradjustment may be made to see if a third instance of the error 1252indicates less distortion than the second instance. The process may berepeated until a subsequent instance of the error 1252 indicates thatthe distortion is below a threshold amount and/or until a certain numberof subsequent instances of the error 1252 do not indicate lessdistortion than a particular instance, which indicates that thedistortion may be minimized. The number of subsequent instances that donot indicate less distortion may be based on a target degree ofminimization of the distortion in which the number may increase as thetolerances for the degree of minimization become stricter.

In these or other embodiments, the training system 1284 may also beconfigured to obtain communication parameters that may correspond to thecommunication of the encoded data 1250 over the communication channel1268 of the network 1202. The communication parameters may includenetwork condition parameters, which may include a type of thecommunication channel 1268 (e.g., cellular line, landline, VoIP line,etc.), loss of the communication channel 1268, noise on thecommunication channel 1268, interference from other signals experiencedon the communication channel 1268, distortion created by thecommunication channel 1268, signal loss, etc. Additionally oralternatively, the communication parameters may include demographicaldata (e.g., age, gender) of a speaker of the audio included in thetraining data 1248 or a characterization of the speaker. Thecharacterization may be determined by extracting features from thespeaker's speech signal and determining one or more parameters thatdescribe the speaker's voice signal. For example, if the speaker has ahigh pitch, the communication parameters may indicate a frequency belowwhich the audio signal need not be transmitted, freeing up bandwidth fortransmitting data. In another example, the speaker's pitch may be usedby the decoder in constructing the decoded audio. In these or otherembodiments, the cost function may use the obtained communicationparameters as inputs such that the training may be based on particularcommunication parameters. As such, the encoding system 1264 and thedecoding system 1266 may be trained and configured to adjust theencoding and the decoding according to different communicationparameters and the error 1252 such that different operations may beperformed according to different communication conditions to reduce orminimize the error 1252. Additionally or alternatively, the decodingsystem 1266 may use the transcription 1060 in decoding audio. Forexample, the decoding system 1266 may convert the transcription 1060 toaudio using a text-to-speech system and use the audio to enhance orreplace the decoded audio.

In some embodiments, the training data 1248 may be altered by theencoding system 1264 such that the encoded data 1250 may besubstantially different from the training data 1248 (e.g., audio of thetraining data 1248 may sound substantially different from audio of theencoded data 1250). However, the decoding system 1266 may be configuredto decode the encoded data 1250 such that the decoded data 1290 issimilar to, substantially the same as, or the same as the training data1248 (e.g., based on the training using the error 1252 discussed above).

In these or other embodiments, the cost function used for training mayinclude terms to encourage the encoding system 1264 to create an audiosignal (e.g., as part of the encoded data 1250) that sounds more likethe live audio signal, possibly with some distortion. The decodingsystem 1266 may be configured to remove the distortion while peopleusing other phones on the communication channel 1268 may still hear theaudio but with the distortion (assuming those phones are not configuredto remove the distortion). In this example, the cost function mayinclude a combination of a first cost function such as function of thedifference between the training data 1248 and the decoded data 1290 anda second cost function that may include a comparison of encoded data1250 to the audio of the training data 1248 that corresponds to theoriginal audio. Such a cost function may cause audio of the encoded data1250 to sound more like the audio of the training data 1248 such that ahearer on another device that does not include the decoding system 1266listening in on a communication session may understand the audioreceived at the other device.

In some embodiments, the encoding system 1264 and the decoding system1266 may include an adaptive network such as a GAN. In these or otherembodiments, the training data 1248 may include particular audio and acorresponding particular transcript, which may be of a currentcommunication session or from one or more previous communicationsessions. FIG. 11 Below is an example of training GANs that may be partof the encoding system 1264 and the decoding system 1266.

In this particular example, the encoding system 1264 may include a firstDNN (DNN1) that encodes the particular audio and the particulartranscription into the encoded data 1250. Additionally or alternatively,in this particular example, the decoding system 1266 may include asecond DNN (DNN2). The decoding system 1266, instead of or in additionto receiving the encoded data 1250, may receive a random data signal asinput. The decoding system 1266 may further receive a set of one or moreadditional communication parameters such as the gender or otherdemographics of the speaker of the particular audio, a characterizationof the speaker of the particular audio, and/or one or more networkcommunication parameters. In these or other embodiments, the additionalparameters may include an adjustment setting that relates to the amountof transcription data the encoding system 1264 may send. In someembodiments, the additional parameters may vary over time or may berelatively constant over the course of a particular call or for aparticular speaker. The decoding system 1266 may use the encoded data1250 and/or the random signal to generate the decoded data 1290, whichmay include the particular audio and the particular transcriptionseparated from each other and reproduced. In this particular example,the combination of the encoding system 1264 and the decoding system 1266may be referred to as a “generator.”

In this particular example, the particular audio of the training data1248 may be communicated to the training system 1254 and the reproducedaudio of the decoded data 1290 may also be communicated to the trainingsystem 1254. In some embodiments, the particular audio of the trainingdata 1248 may be filtered (e.g., using the first filter 1176 of FIG. 11) prior to being sent to the training system 1254. In these or otherembodiments, the filtering of the particular audio may modify thetraining of the decoding system 1266.

For example, the filtering may attenuate audio at certain frequencies(for example, by attenuating the signal above a specified frequency suchas 3.6 kHz) so that the decoding system 1266 is trained to attenuate thecorresponding frequencies. The filter used to perform the filtering maybe a classical finite impulse response or infinite impulse responsefilter or it may be included in or implemented by a neural network ofthe encoding system 1264. In some embodiments, the parameters of thefilter may be responsive to an audiogram or cochlear implant MAP basedon the hearing or hearing device of a recipient of the audio (e.g., of auser of a device participating in the communication session).

In some embodiments, DNN1 and DNN2 may be trained using adversarialtraining. For example, the training system 1254 may be configured toselect, at random (e.g., using a switch) between the particular audio ofthe training data 1248 and the reproduced audio of the decoded data1290. In these or other embodiments, the training system 1254 mayinclude a discriminator (e.g., a third DNN (DNN3)) that guesses whetherthe selected audio is the particular audio or the reproduced audio. Inthese or other embodiments, the guess may be used as a first trainingsignal and may be used to train the generator (e.g., DNN1 and DNN2 ofthe encoding system 1264 and the decoding system 1266, respectively).The generator may be trained to generate the reproduced audio such thatthe reproduced audio is selected by the discriminator as often aspossible. Alternatively or additionally, the generator may be trained togenerate the reproduced audio that is as close as possible to theoriginal data signal and party 2 audio. Alternatively or additionally,the training may be based on multiple training objectives (e.g., (1) thediscriminator selects the reproduced audio and (2) the reproduced audiois close to the original). In these or other embodiments, the trainingmay be a combination of the objectives such as a sum or weighted sum.

In these or other embodiments, the guess from the discriminator may becompared to the selected audio by a comparator to create a secondtraining signal. For example, in some embodiments, the guess from thediscriminator may be a binary value where a zero may represent a guessthat the selected audio is the particular audio and where a one mayrepresent a guess that the selected audio is the reproduced audio, orvice versa. In these or other embodiments, the selected audio may alsohave a binary value that represents whether it is the reproduced audioor the particular audio. The comparator may be configured to determinewhether the two values match to determine whether the discriminator madea correct guess. The output of the comparator may be based on whetherthe two values match and may be used as the second training signal. Thesecond training signal may be used to train the discriminator. Thesecond training signal may be further used to train the generator. Thegenerator and discriminator may be trained simultaneously (all weightsare trained at the same time) or alternately (meaning that thediscriminator weights are held constant while the generator weights aretrained and vice versa).

The above description of using adversarial training to train elementssuch as the encoding system 1264 and the decoding system 1266 isillustrative. Other training techniques may be used. For example,training the encoding system 1264 and the decoding system 1266 may use aloss function that is a sum or weighted sum of the discriminator error,the difference between the particular audio and the reproduced audio,and difference between the particular audio and the encoded audio of theencoded data 1250 that corresponds to the particular audio. In anotherexample, the discriminator may produce additional outputs that indicate,for example, the particular audio, the discriminator input, or ahuman-labeled classification of the discriminator input. In anotherexample, training may include adversarial training combined (e.g., usinga summed loss function or by training alternately) with loss functionsand other training techniques described above. As another example,although the above training is described in the context of theparticular audio of the training data 1248 and the reproduced audio ofthe decoded data 1290, similar operations may be performed to train thegenerator with respect to the particular transcription of the trainingdata 1248 and the corresponding reproduced transcription of the decodeddata 1290.

The neural networks described may have any number of differentconnections and architectures. Below are some examples of portions ofthose connections and architectures. For example, in some embodiments, aparticular neural network may have an input layer and one or more hiddenlayers. The input layer may be a series of input samples such as audiosamples. The next layer (the first hidden layer) may be a firstfully-connected layer. The next layer may be a second fully-connectedlayer. Alternatively or additionally, the neural networks may includeother types of layers such as recurrent layers, convolutional layers,and pooling layers after each convolutional layer. In these or otherembodiments, one or more input samples may be taken from a correspondingoutput sample. For instance, an example neural network may take inputfrom a first audio signal and a second audio signal. The first audiosignal may include a series of n input audio samples s₁, s, . . . ,s_(n), of audio that is to be encoded and the second audio signal may bethe previous n output audio samples, o₁, o, . . . , o_(n) of the neuralnetwork.

Additionally or alternatively, in some embodiments, the neural networksmay be pruned to reduce the number of connections included in the neuralnetworks, which may increase the speed of training and/or require lesstraining data. For instance, in an example neural network that includestwo convolutional layers, the input samples of an input signal (e.g., anaudio signal) may pass through the two convolutional layers and may thenbe reduced to a single output sample by a last output layer. In someembodiments, the neural network may have a dilated convolution where aconvolution filter skips a number (referred to as the dilation number)of inputs of the convolution layers. For example, in the firstconvolution layer, no inputs are skipped. In the second convolutionlayer (dilation=2), alternate inputs may be skipped. In the output layer(dilation=4), three inputs are skipped. Although a dilation that doubleswith each subsequent layer is described, other dilation rates arepossible such as tripling or quadrupling the dilation with each layer.Dilated convolution may be used for the neural networks described hereinthat output audio samples.

The environment 1200 may accordingly be configured to train neuralnetworks that may be used to encode and decode audio and correspondingtranscriptions for the communicating of the audio and correspondingtranscriptions over a same communication channel, such as a same phoneline. Modifications, additions, or omissions may be made to theenvironment 1200 and/or the components operating in the environment 1200without departing from the scope of the present disclosure. For example,in some embodiments, the environment 1200 may be integrated into otherenvironments that provide additional benefits for a user. As anotherexample, the particular arrangement and description of the componentsare merely examples used to help explain the concepts described hereinand are not meant to be limiting.

Further, as indicated above, the encoding system 1264 and the decodingsystem 1266 may be configured as an autoencoder in some embodiments.FIG. 12B illustrates an example autoencoder 1120 that may include anencoding system 1265 (which may be an example of the encoding system1264 of FIG. 12A) and a decoding system 1267 (which may be an example ofthe decoding system 1266 of FIG. 12A). In these or other embodiments,the autoencoder 1220 may be configured to encode audio 1262 and atranscription 1260 into encoded data 1251. The audio 1262 and thetranscription 1260 may be the training data 1248 in some embodiments.Additionally or alternatively, the audio 1262 may be analogous to theaudio 1162 of FIG. 11 . In these or other embodiments, the transcription1260 may be analogous to the transcription 1160 of FIG. 11 .

In some embodiments, the encoded data 1251 may be an example of thecombined data 1150 of FIG. 11 . In some embodiments, the encoded data1251 may be encoded such that the encoded audio 1262 and the encodedtranscription 1260 occupy overlapping communication resources (e.g.,overlapping frequency bands and/or time slots). Additionally oralternatively, the encoded data 1251 may be encoded such that theencoded audio 1262 and the encoded transcription 1260 occupy differentcommunication resources (e.g., different frequency bands and/or timeslots).

In the illustrated example, the encoding system 1265 may include a firstshift register 1222 configured to receive the audio 1262. Additionally,in the illustrated example, the encoding system 1265 may include a modem1226 configured to modulate the transcription 1260 onto a data signal,which may be communicated to a second shift register 1224. The outputsof the first shift register 1222 and the second shift register 1224 maybe received at input nodes 1242 of a first neural network 1240 of theencoding system 1265.

The first neural network 1240 may be configured to encode the audio 1262and the transcription 1260 to generate the encoded data 1251 byperforming any suitable processing operation on the data received at theinput nodes 1242. In these or other embodiments, the encoded data 1251may be output at an output node 1244 of the first neural network 1240.As illustrated in FIG. 12B, the number of input nodes 1242 (illustratedby way of example as being fourteen input nodes) may be greater than thenumber of output nodes 1244 (illustrated by way of example as being oneoutput node).

The encoded data 1251 may be communicated to the decoding system 1267via the network 1202 which is illustrated in both FIGS. 12A and 12B. Thedecoding system 1267 may include a third shift register 1228 that isconfigured to receive the encoded data 1251. The third shift register1228 may be an “n” bit (in the illustrated example 15 bit) shiftregister that may be communicatively coupled to input nodes 1232 of asecond neural network 1246 of the decoding system 1267. The third shiftregister 1228 may be configured such that the encoded data 1251 receivedat the third shift register 1228 is communicated to the input nodes 1232of the second neural network 1246.

The second neural network 1246 may be configured to perform one or moreprocessing operations on the encoded data 1251, as received at the inputnodes 1232, to decode the encoded data 1251 to separate the audio 1262and the transcription 1260 of the encoded data 1251. In someembodiments, the second neural network 1246 may output the separatedaudio as reproduced audio 1262 at a first output node 1234. In these orother embodiments, the second neural network 1246 may output theseparated transcription at a second output node 1236. In someembodiments, such as those in which the first encoding system 1265includes the modem 1226, the separated transcription as output by thesecond output node 1236 may still be modulated on the data signal. Inthese or other embodiments, the decoding system 1267 may include a modem1230 configured to demodulate the transcription output at the secondoutput node 1236 and output the demodulated transcription as areproduced transcription 1260. The reproduced audio 1262 and reproducedtranscription 1260 may be examples of the decoded data 1290 of FIG. 12A.

Modifications, additions, or omissions may be made to the autoencoder1220 without departing from the scope of the present disclosure. Forexample, the number of input nodes and/or output nodes and the number ofneural network layers of the encoding system 1256 and/or of the decodingsystem 1267 may vary. Additionally, the neural network configuration mayvary and may include other topologies such as recurrent layers.Additionally, the second neural network 1246 may be implemented as twoneural networks, one with a first output node 1234 and another with asecond output node 1236. Further, the modem 1226 and/or the modem 1230may be omitted in some embodiments. Additionally, the configurations,sizes, etc., of the shift registers may vary. Moreover, in someembodiments, one or more of the shift registers may be omitted.Additionally or alternatively, the first shift register 1222 and thesecond shift register 1224 may be combined into a single shift register.

FIG. 13 is a flowchart of an example method 1300 to communicate atranscription and corresponding audio over a same communication channel.The method 1300 may be arranged in accordance with at least oneembodiment described in the present disclosure. One or more of theoperations of the method 1300 may be performed, in some embodiments, bya device or system, such as the transcription systems of any of theabove FIGS., the first signal processing system 1064 and the secondsignal processing system 1066 of FIG. 10 , the first signal processingsystem 1164 and the second signal processing system 1166 of FIG. 11 ,the encoding system 1264 and the decoding system 1266 of FIG. 12A, theencoding system 1265 and the decoding system 1267 of FIG. 12B, or thecomputing system 1400 of FIG. 14 , or any other suitable another deviceor system. In these and other embodiments, the method 1300 may beperformed based on the execution of instructions stored on one or morenon-transitory computer-readable media. Although illustrated as discreteblocks, various blocks may be divided into additional blocks, combinedinto fewer blocks, or eliminated, depending on the desiredimplementation.

The method 1300 may begin at block 1302, where audio originating at aremote device during a communication session conducted between a firstdevice and the remote device may be obtained. In some embodiments, theaudio may be obtained by a transcription system via any suitableoperation described above with respect to FIGS. 9A and 9B. At block1304, a transcription of the audio may be obtained.

At block 1306, the audio may be processed to generate processed audio.The processing of the audio may include one or more of the audioprocessing operations described above with respect FIGS. 10, 11 and 12in some embodiments. In some embodiments, the processing of the audiomay be performed by a neural network such that the audio may be encodedby the neural network. In these or other embodiments, the neural networkmay be trained with respect to a voice network, such as an analog voicenetwork. In some embodiments, the training may include one or moretraining operations described above with respect to FIG. 12 . Asdiscussed above, in some embodiments, the audio may be processed suchthat the processed audio uses a first communication resource of acommunication channel of the voice network (e.g., a phone line) andleaves a second communication resource of the same communication channelavailable for communication of the processed transcription.

At block 1308, the transcription may be processed to generate aprocessed transcription. The processing of the transcription may includeone or more of the transcription processing operations described abovewith respect FIGS. 10, 11 and 12 in some embodiments. In someembodiments, the processing of the transcription may be such that theprocessed transcription is formatted for communication over the voicenetwork. For example, as discussed above with respect to FIGS. 10 and 11, the transcription may be modulated by a modem and or converted into anaudio data signal (e.g., using DTMF signaling) for communication overthe voice network. In some embodiments, the processing of thetranscription may be performed by a neural network such that thetranscription may be encoded by the neural network. Additionally oralternatively, in some embodiments, the transcription may be processedsuch that the processed transcription uses the second communicationresource of the communication channel.

At block 1310, the processed audio may be multiplexed with the processedtranscription to obtain combined data. The multiplexing of the processedaudio and the processed transcription may include performing one or moreof the operations described above with respect to FIGS. 10 and 11 toobtain the combined data 1050 and 1150. As discussed above, in someembodiments the multiplexing may be such that the processed audio andthe processed transcription may use a same communication resource of thecommunication channel of the voice network. In these or otherembodiments, the multiplexing may be such that the processed audio andthe processed transcription may use different communication resources(e.g., the first communication resource and the second communicationresource) of the same communication channel of the voice network. Forexample, the multiplexing may include time multiplexing and/or bandwidthmultiplexing such as described above such that the processed audio andthe processed transcription of the combined data use different timeslots and/or frequencies. Additionally or alternatively, themultiplexing may use carrierless amplitude phase modulation, quadratureamplitude modulation, code division multiplexing, time divisionmultiplexing, spread spectrum and methods that combine processed audioand processed transcriptions into overlapping time slots and/orfrequencies. In these or other embodiments, the multiplexing may bebased on the first communication resource and/or the secondcommunication resource such that the processed audio of the combineddata uses the first communication resource and the processedtranscription of the combined data uses the second communicationresource.

At block 1312, the combined data may be communicated to the first deviceduring the communication session. As indicated above, in someembodiments, the combined data may be communicated using the samecommunication channel of the voice network. Further, as indicated above,the combined data may be communicated such that the processed audio andthe processed transcription are communicated using the samecommunication resource or using different communication resources.

It is understood that, for this and other processes, operations, andmethods disclosed herein, the functions and/or operations performed maybe implemented in differing order. Furthermore, the outlined functionsand operations are only provided as examples, and some of the functionsand operations may be optional, combined into fewer functions andoperations, or expanded into additional functions and operations withoutdetracting from the essence of the disclosed embodiments. For example,in some embodiments, the method 1300 may further include one or moreoperations described above with respect to identifying and reproducingthe audio and the transcription from the combined data (e.g., decodingthe combined data) and presenting the reproduced audio and transcriptionsuch as described with respect to the second signal processing systems1066 and 1166 of FIGS. 10 and 11 and the decoding system 1266 of FIG. 12.

FIG. 14 illustrates an example system 1400 that may be used duringtransfer of communication between devices as described in thisdisclosure. The system 1400 may include a processor 1410, memory 1412, acommunication unit 1416, a display device 1418, a user interface unit1420, and a peripheral device 1422, which all may be communicativelycoupled. In some embodiments, the system 1400 may be part of any of thesystems or devices described in this disclosure.

For example, the system 1400 may be part of the environment 100 of FIG.1 and may be configured to perform one or more of the tasks describedabove with respect to the first device 112. As another example, thesystem 1400 may be part of the environment of FIG. 2 and may beconfigured to perform one or more of the tasks described above withrespect to the first device 212, the second device 214, or thetranscription system 230. As another example, the system 1400 may bepart of the environment 500 of FIG. 5 and may be configured to performone or more of the tasks described above with respect to the supportsystem 520. As another example, the system 1400 may be part of theenvironment 800 of FIG. 8 and may be configured to perform one or moreof the tasks described above with respect to the monitor system 820. Asanother example, the system 1400 may be part of the environment 900 ofFIG. 9 a and may be configured to perform one or more of the tasksdescribed above with respect to the presentation system 906. As anotherexample, the system 1400 may be part of the environment 1000 of FIG. 10and may be configured to perform one or more of the tasks describedabove with respect to the presentation system 1006.

Generally, the processor 1410 may include any suitable special-purposeor general-purpose computer, computing entity, or processing deviceincluding various computer hardware or software modules and may beconfigured to execute instructions stored on any applicablecomputer-readable storage media. For example, the processor 1410 mayinclude a microprocessor, a microcontroller, a digital signal processor(DSP), an application-specific integrated circuit (ASIC), aField-Programmable Gate Array (FPGA), graphics processing unit (GPU),vector or array processor, a SIMD (single instruction multiple data) orother parallel processor, or any other digital or analog circuitryconfigured to interpret and/or to execute program instructions and/or toprocess data.

Although illustrated as a single processor in FIG. 14 , it is understoodthat the processor 1410 may include any number of processors distributedacross any number of networks or physical locations that are configuredto perform individually or collectively any number of operationsdescribed herein. In some embodiments, the processor 1410 may interpretand/or execute program instructions and/or process data stored in thememory 1412. In some embodiments, the processor 1410 may execute theprogram instructions stored in the memory 1412.

For example, in some embodiments, the processor 1410 may execute programinstructions stored in the memory 1412 that are related to operationsfor generating transcriptions such that the system 1400 may perform ordirect the performance of the operations associated therewith asdirected by the instructions.

The memory 1412 may include computer-readable storage media or one ormore computer-readable storage mediums for carrying or havingcomputer-executable instructions or data structures stored thereon. Suchcomputer-readable storage media may be any available media that may beaccessed by a general-purpose or special-purpose computer, such as theprocessor 1410.

By way of example, and not limitation, such computer-readable storagemedia may include non-transitory computer-readable storage mediaincluding Random Access Memory (RAM), Read-Only Memory (ROM),Electrically Erasable Programmable Read-Only Memory (EEPROM), flashmemory, Compact Disc Read-Only Memory (CD-ROM) or other optical diskstorage, magnetic disk storage or other magnetic storage devices, flashmemory devices (e.g., solid state memory devices), or any other storagemedium which may be used to carry or store particular program code inthe form of computer-executable instructions or data structures andwhich may be accessed by a general-purpose or special-purpose computer.Combinations of the above may also be included within the scope ofcomputer-readable storage media.

Computer-executable instructions may include, for example, instructionsand data configured to cause the processor 1410 to perform a certainoperation or group of operations as described in this disclosure. Inthese and other embodiments, the term “non-transitory” as explained inthe present disclosure should be construed to exclude only those typesof transitory media that were found to fall outside the scope ofpatentable subject matter in the Federal Circuit decision of In reNuijten, 500 F.3d 1346 (Fed. Cir. 2007). Combinations of the above mayalso be included within the scope of computer-readable media.

The communication unit 1416 may include any component, device, system,or combination thereof that is configured to transmit or receiveinformation over a network. In some embodiments, the communication unit1416 may communicate with other devices at other locations, the samelocation, or even other components within the same system. For example,the communication unit 1416 may include a modem, a network card(wireless or wired), an infrared communication device, a wirelesscommunication device (such as an antenna), and/or chipset (such as aBluetooth device, an 802.6 device (e.g., Metropolitan Area Network(MAN)), a WiFi device, a WiMax device, cellular communicationfacilities, etc.), a telephone jack, and/or the like. The communicationunit 1416 may permit data to be exchanged with a network and/or anyother devices or systems described in the present disclosure.

The display device 1418 may be configured as one or more displays thatpresent images, words, etc., like an LCD, LED, OLED, projector, or othertype of display. The display device 1418 may be configured to presentvideo, text captions, user interfaces, and other data as directed by theprocessor 1410. For example, when the system 1400 is included in thefirst device 112 of FIG. 1 , the display device 1418 may be configuredto present transcriptions.

The user interface unit 1420 may include any device to allow a user tointerface with the system 1400. For example, the user interface unit1420 may include a mouse, a track pad, a keyboard, buttons, and/or atouchscreen, among other devices. The user interface unit 1420 mayreceive input from a user and provide the input to the processor 1410.In some embodiments, the user interface unit 1420 and the display device1418 may be combined.

The peripheral devices 1422 may include one or more devices. Forexample, the peripheral devices may include a microphone, an imager,and/or a speaker, among other peripheral devices. In these and otherembodiments, the microphone may be configured to capture audio. Theimager may be configured to capture images. The images may be capturedin a manner to produce video or image data. In some embodiments, thespeaker may present audio received by the system 1400 or otherwisegenerated by the system 1400 by broadcasting the audio.

Modifications, additions, or omissions may be made to the system 1400without departing from the scope of the present disclosure. For example,in some embodiments, the system 1400 may include any number of othercomponents that may not be explicitly illustrated or described. Further,depending on certain implementations, the system 1400 may not includeone or more of the components illustrated and described.

As indicated above, the embodiments described herein may include the useof a special purpose or general purpose computer (e.g., the processor1410 of FIG. 14 ) including various computer hardware or softwaremodules, as discussed in greater detail below. Further, as indicatedabove, embodiments described herein may be implemented usingcomputer-readable media (e.g., the memory 1412 of FIG. 14 ) for carryingor having computer-executable instructions or data structures storedthereon.

In some embodiments, the different components, modules, engines, andservices described herein may be implemented as objects or processesthat execute on a computing system (e.g., as separate threads). Whilesome of the systems and methods described herein are generally describedas being implemented in software (stored on and/or executed by generalpurpose hardware), specific hardware implementations or a combination ofsoftware and specific hardware implementations are also possible andcontemplated.

In accordance with common practice, the various features illustrated inthe drawings may not be drawn to scale. The illustrations presented inthe present disclosure are not meant to be actual views of anyparticular apparatus (e.g., device, system, etc.) or method, but aremerely idealized representations that are employed to describe variousembodiments of the disclosure. Accordingly, the dimensions of thevarious features may be arbitrarily expanded or reduced for clarity. Inaddition, some of the drawings may be simplified for clarity. Thus, thedrawings may not depict all of the components of a given apparatus(e.g., device) or all operations of a particular method.

Terms used herein and especially in the appended claims (e.g., bodies ofthe appended claims) are generally intended as “open” terms (e.g., theterm “including” should be interpreted as “including, but not limitedto,” the term “having” should be interpreted as “having at least,” theterm “includes” should be interpreted as “includes, but is not limitedto,” etc.).

Additionally, if a specific number of an introduced claim recitation isintended, such an intent will be explicitly recited in the claim, and inthe absence of such recitation no such intent is present. For example,as an aid to understanding, the following appended claims may containusage of the introductory phrases “at least one” and “one or more” tointroduce claim recitations. However, the use of such phrases should notbe construed to imply that the introduction of a claim recitation by theindefinite articles “a” or “an” limits any particular claim containingsuch introduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitationis explicitly recited, it is understood that such recitation should beinterpreted to mean at least the recited number (e.g., the barerecitation of “two recitations,” without other modifiers, means at leasttwo recitations, or two or more recitations). Furthermore, in thoseinstances where a convention analogous to “at least one of A, B, and C,etc.” or “one or more of A, B, and C, etc.” is used, in general such aconstruction is intended to include A alone, B alone, C alone, A and Btogether, A and C together, B and C together, or A, B, and C together,etc. For example, the use of the term “and/or” is intended to beconstrued in this manner.

Further, any disjunctive word or phrase presenting two or morealternative terms, whether in the description, claims, or drawings,should be understood to contemplate the possibilities of including oneof the terms, either of the terms, or both terms. For example, thephrase “A or B” should be understood to include the possibilities of “A”or “B” or “A and B.”

Additionally, the use of the terms “first,” “second,” “third,” etc., arenot necessarily used herein to connote a specific order or number ofelements. Generally, the terms “first,” “second,” “third,” etc., areused to distinguish between different elements as generic identifiers.Absence a showing that the terms “first,” “second,” “third,” etc.,connote a specific order, these terms should not be understood toconnote a specific order. Furthermore, absence a showing that the termsfirst,” “second,” “third,” etc., connote a specific number of elements,these terms should not be understood to connote a specific number ofelements. For example, a first widget may be described as having a firstside and a second widget may be described as having a second side. Theuse of the term “second side” with respect to the second widget may beto distinguish such side of the second widget from the “first side” ofthe first widget and not to connote that the second widget has twosides.

All examples and conditional language recited herein are intended forpedagogical objects to aid the reader in understanding the invention andthe concepts contributed by the inventor to furthering the art, and areto be construed as being without limitation to such specifically recitedexamples and conditions. Although embodiments of the present disclosurehave been described in detail, it should be understood that the variouschanges, substitutions, and alterations could be made hereto withoutdeparting from the spirit and scope of the present disclosure.

We claim:
 1. A method to access a device, the method comprising:obtaining, at a first device, data over a short-range wireless networkfrom a second device, the data originating at a remote system that sendsthe data to the second device through a network connection over a widearea network; in response to a fault at the second device, obtaining, atthe first device from the remote system, a maintenance command for thesecond device, the maintenance command obtained by the first device overan analog voice network; and directing, from the first device to thesecond device, the maintenance command over the short-range wirelessnetwork to enable the second device to perform the maintenance command.2. The method of claim 1, wherein the data is a transcription of audioobtained by the first device over the analog voice network during acommunication session between the first device and a remote device. 3.The method of claim 2, further comprising directing, from the firstdevice to the remote system, the audio by way of the short-rangewireless network, the second device, and the wide area network, theremote system configured to generate the transcription using the audio.4. The method of claim 1, wherein the fault at the second deviceincludes an issue with respect to the network connection over the widearea network between the remote system and the second device.
 5. Themethod of claim 4, wherein the issue with respect to the networkconnection over the wide area network between the remote system and thesecond device is a failure of the network connection.
 6. The method ofclaim 1, further comprising: in response to providing the maintenancecommand to the second device, obtaining a response from the seconddevice with respect to the maintenance command; and directing theresponse to the remote system over the analog voice network.
 7. Themethod of claim 1, wherein the fault is detected by the remote system.8. The method of claim 1, further comprising: detecting, by the firstdevice, the fault in the second device; and providing an indication ofthe fault to the remote system over the analog voice network, whereinthe maintenance command is obtained in response to providing theindication of the fault to the remote system.
 9. The method of claim 1,wherein the short-range wireless network is a personal area network oran 802.11 network, the wide area network includes one or more of: acellular network, a digital network, and an optical network, and theanalog voice network is a plain old telephone system network.
 10. Themethod of claim 1, wherein the maintenance command relates to one ormore of the following: parameters for connection over the wide areanetwork, firewall settings, firmware updates, resetting commands,configuration settings, and settings of the short-range wirelessnetwork.
 11. At least one non-transitory computer-readable mediaconfigured to store one or more instructions that, in response to beingexecuted by at least one processor, cause or direct a system to performthe method of claim
 1. 12. A system comprising: a memory configured tostore instructions; and one or more hardware processors coupled to thememory and configured to execute the instructions to cause or direct thesystem to perform operations, the operations comprising: obtain, at afirst device, data over a short-range wireless network from a seconddevice, the data originating at a remote system that sends the data tothe second device through a network connection over a wide area network;in response to a fault at the second device, obtain, at the first devicefrom the remote system, a maintenance command for the second device, themaintenance command obtained by the first device over an analog voicenetwork; and direct, from the first device to the second device, themaintenance command over the short-range wireless network to enable thesecond device to perform the maintenance command.
 13. The system ofclaim 12, wherein the data is a transcription of audio obtained by thefirst device over the analog voice network during a communicationsession between the first device and a remote device.
 14. The system ofclaim 13, wherein the operations further comprise direct, from the firstdevice to the remote system, the audio by way of the short-rangewireless network, the second device, and the wide area network, theremote system configured to generate the transcription using the audio.15. The system of claim 12, wherein the fault at the second deviceincludes an issue with respect to the network connection over the widearea network between the remote system and the second device.
 16. Thesystem of claim 15, wherein the issue with respect to the networkconnection over the wide area network between the remote system and thesecond device is a failure of the network connection.
 17. The system ofclaim 12, wherein the operations further comprise: in response toproviding the maintenance command to the second device, obtain aresponse from the second device with respect to the maintenance command;and direct the response to the remote system over the analog voicenetwork.
 18. The system of claim 12, wherein the operations furthercomprise: detect, by the first device, the fault in the second device;and provide an indication of the fault to the remote system over theanalog voice network, wherein the maintenance command is obtained inresponse to providing the indication of the fault to the remote system.19. The system of claim 12, wherein the short-range wireless network isa personal area network or an 802.11 network, the wide area networkincludes one or more of: a cellular network, an optical network, and theanalog voice network is a plain old telephone system network.
 20. Thesystem of claim 12, wherein the maintenance command relates to one ormore of the following: parameters for connection over the wide areanetwork, firewall settings, firmware updates, resetting commands,configuration settings, and settings of the short-range wirelessnetwork.